1	The University of Alabama Online Profile System (UAOPS) Administrative Unit Profiles Workshop 2007/2008
2	UAOPS Is a publicly accessible online data management system composed of relational databases. Has academic and administrative unit profiles, course information, and faculty teaching credentials. Documents annual assessment reporting by unit to support the University’s accountability for assessment.
3	UAOPS is Publicly Accessible All information in UAOPS except certain management reports can be viewed without a password. The UAOPS Homepage has clearly marked read-only sections where all information can be viewed.
4	The University of Alabama Homepage http://www.ua.edu
5	UAOPS Homepage
6	Who Uses UAOPS? Regional and program specific accrediting organizations. UA faculty, staff and administrators such as the Provost and faculty advisors Students, prospective students and their families.
7	Why Update Annually? Mandated by SACS Principles of Accreditation 3.3.1
8	Why The Emphasis on Correct Writing? Revision is Needed Overall, there was a significant improvement in the administrative unit profiles last year. The most common areas needing improvement were: Each expected outcome should be a significant component of the unit’s mission. Some expected outcomes contained multiple expectations, each needing assessment. Many unit profiles lacked summaries of assessment methods and/or results, a data summary, or they were not clearly worded. Not following the 1 to 1 to 1 correspondence rule.
9	Need for Consistency Throughout Organization The University of Alabama Mission The Division’s Mission Statement The Unit’s Mission Statement
10	Need for Consistency Between Various Documents Throughout Organization The Unit Profile mission statement should be the same as that shown on the administrative unit website. The Unit Profile should represent the most important features of the more detailed Administrative Unit Strategic Plan. There should be congruency between the UAOPS Unit Profile and any assessment plans prepared for the Administrative Unit.
11	UAOPS Administrative Unit Profile New for 2007/2008: 4-Column Format
12	Also New for 2007/2008 Archived unit profiles from 2005 and 2006 can be viewed by selecting the year. This year, unit profiles will be completed on a prospective basis The outcomes for last year are populated into the grid for 2007/2008, but the remaining areas are blank
13	The New Process for 2007/2008 The 2007/2008 outcomes in column 1 should be edited, changed, or deleted as needed now. The assessment methods planned for those outcomes should be described in column 2 entitled, “Unit Methods of Assessment”. As results are available, they should be added to column 3, entitled, “Unit Results of Assessment”. Please include a data summary. When the assessment results have been reviewed and formally discussed by staff and management, and changes and improvements have been implemented, that information should be summarized in column 4. This last step will occur after the spring semester. The deadline for completion of administrative unit profiles is June 1, 2008. This date was chosen to allow most units to finalize the unit profile along with their annual report. Unit Profiles will be evaluated after June 1 and archived August 20, 2008. An evaluation report will be emailed to each unit.
14	What is Required for Administrative and Support Units? To answer the question, How well is the unit providing services to its individual group of clients?
15	Suggested Steps for Administrative Units Consider how the unit’s mission statement reflects one or more of the primary components of UA’s mission statement. Establish the administrative unit’s mission statement, listing all services provided. Formulate the administrative unit’s expected outcomes based on the services listed in the unit’s mission statement. Identify unit methods of assessment and criteria for success Conduct assessment activities. Discuss and document use of results for service improvements. Source: Nichols and Nichols, 2005
16	The University of Alabama Mission Statement The University of Alabama 2006/2007 FACTBOOK, page 3 http://bama.ua.edu/~oir/factbook/
17	Establish the Administrative Unit’s Service- Related Mission Statement May include a philosophical statement of purpose. Needs to include a list of services provided by the unit. Needs to be revisited at least annually to determine if any services have been added or are no longer provided.
18	Process: Constructing Administrative Outcomes Look at the services listed in the unit mission statement of unit. Assessment should seek these answers: What is the skill/knowledge/benefit that we want our clients to receive from the current services offered? Are the students or other clients learning/benefiting from the current services offered? What is the level of satisfaction of our clients with current services?
19	Keep It Current Outcomes should focus on currently existing services as described in the unit mission statement. They should not focus on administrative planning activities that are future focused. (Continued)
20	Keep It Current If the only answer to the assessment question is, “yes, they did it”, or, “no, they didn’t do it”, then, the service is probably a planned one, not a current one.
21	Long List/Short List Process: Constructing Administrative Outcomes The Unit’s staff : Reviews services as described in the mission statement. Creates a long list of objectives/outcomes related to those services. From that list, staff selects outcomes to be assessed during upcoming assessment cycle.
22	How Many Administrative Outcome Statements are Needed? Recommendations Vary Some experts recommend 5 to 8 expected outcome statements Use of only 1 outcome statement could raise the question of whether unit takes assessment seriously. If there are many more than 5 to 8, there will be time for little else.
23	Make sure that an outcome is related to something under the control of the unit. Better: Once grades are received, the registrar's office will decrease the time for posting of grades by _____. Good: Registrar’s office wants to improve time for posting grades to final transcripts.
24	Example Outcomes (Name of Service) (Name of Unit) will
25	Example Outcomes (Service) will increase overall client satisfaction rating to an average score of 4.3 out of a possible 6.0.
26	Example Outcomes
27	Assessment of Expected Outcomes Each Expected Outcome must be assessed. Methods and results that include data should be clearly summarized. Choose multiple methods for assessment, both direct and indirect whenever possible or appropriate. Direct measures directly measure the outcome; indirect measures do not directly measure the outcome.
28	Methods of Assessment (Column 2) Direct Assessment Technique Examples Volume of activity such as number of persons served. Levels of efficiency such as the average time for response. Measures of quality such as average errors per audit
29	“Campus Events”, a Hypothetical Direct Assessment Technique Example Informal Method of Assessment Can Become a Formal Method of Assessment Informal observation or report of service not working as it should, such as a student event. Discussion at a staff meeting/brainstorming ways to correct problem or use different approach. Staff members formulate a desired outcome such as, “Students will enjoy the night activities planned by Campus Events”. Source: Nichols and Nichols, 2005
30	Students Will Enjoy the Night Activities Planned by Campus Events Staff develops method (a rubric) to assess this outcome. A rubric is a table that lists specific traits that apply to the skill being taught and defines quality levels for each trait Makes list of several factors expected to be seen if students are enjoying an event such as: showing an interest, smiling, attending event, expressing gratitude, and specific comments made by students
31	Campus Events Rubric Develop checklist or scoring rubric for evaluating the campus event using above factors. Define a threshold for success: For example, an overall average score of 3.6 will be received by all events, with no individual event scoring less than 2.5. Have more than one staff member attend future events and uses the scoring rubric for recording observations of student enjoyment of the event and any comments made.
32	Students Will Enjoy the Night Activities Planned by Campus Events Results are tabulated: After each event, at least 2 staff members tabulate their individual ratings of student enjoyment of the event and list any specific comments made by students. Overall scores are tabulated for each event based on the staff members ratings.
33	Students Will Enjoy the Night Activities Planned by Campus Events Improvements/Changes (Column 4) are planned Assessment results data are discussed at staff meeting and/or annual retreat Improvements/Changes based on assessment data are implemented
34	UAOPS Academic Unit Profile Campus Events Example (Hypothetical)
35	The Update Process (In brief)
36	Who Can Update UAOPS? Each unit has a representative or “contact” with a username and password. The contact can update or upload information. Unit Profile updates are entered directly into the text display area.
37	Instructions for Updates are Available on the UAOPS Website Homepage
38	Privacy Considerations UAOPS users and department heads should use care that no protected individual information is placed on the website. Examples include social security numbers, birthdates, data on identifiable students, or other protected information.
39	Information on Privacy Rules The University of Alabama website has Family Educational Rights and Privacy Act (FERPA) training presentations available. FERPA 101 is for UA employees. FERPA 102 is for administrators
40	Frequently Asked Questions A list of FAQs relating to use of UAOPS is posted at the UAOPS website. The Institutional Research and Assessment staff are available to help at any time.
41	The Process for 2007/2008 Edit and finalize the current 2007/2008 outcomes in column 1 now. The assessment methods planned for those outcomes should be summarized in column 2 entitled, “Unit Methods of Assessment”. As results are available, they should be added to column 3, entitled, “Unit Results of Assessment”. Include a data summary. When the assessment results have been reviewed and formally discussed by faculty and management, and curricular changes and improvements have been implemented, summarize that information in column 4. This last step will occur after the spring semester. The final deadline for completion of everything is the August 20, 2008 archive date. This date was chosen to allow most units to finalize the unit profile along with their annual report.
42	Feedback to Units When draft unit profiles are completed, they will be evaluated. Feedback reports will be sent to the units again this year. Profiles can be edited until the archive date of August 20, even after the report is submitted.
43	Annual Archiving of Results Each year, in August, the completed unit profiles are archived to document progress and the University’s commitment to ongoing assessment at the unit level. The feedback reports will be archived along with the unit profile.
44	Questions? This concludes the material on writing unit profiles. A short technical demonstration follows if you are interested. If you do not need the technical demonstration, you are dismissed.
45	Thank you.
46	SOURCES The SACS-COC 2006 Annual Meeting The 2006 Annual Meeting of the Commission on Colleges NPEC Assessment Sourcebook, 2005 Nichols, James O., and Nichols, Karen W., A ROAD MAP FOR IMPROVEMENT OF STUDENT LEARNING AND SUPPORT SERVICES THROUGH ASSESSMENT, Agathon Press, NY, 2005. “Writing Student Outcomes”, University of Southern California, www.usc.edu, 2006. Walvoord, Barbara E., ASSESSMENT CLEAR AND SIMPLE, Jossey-Bass, 2004.