1	The University of Alabama Online Profile System (UAOPS) Academic Unit Profiles Workshop 2007/2008
2	Subject Matter of Session Writing Academic Profiles Correctly Technical aspects/mechanics of posting a profile (minor focus) Will not focus on uploading course syllabi and faculty CV’s. Any questions can be handled after the workshop
3	What is UAOPS? An online data management system composed of relational databases. UAOPS manages and displays information unit profiles, faculty teaching credentials, and course information.
4	Background of UAOPS: Began as component of SACS Website during the 2005 reaffirmation of the University. Recreated in 2006 to provide a website where units can continually document annual updates of unit profiles, teaching credentials, and course information.
5	UAOPS is Publicly Accessible All information in UAOPS except certain management reports can be viewed without a password. The UAOPS Homepage has clearly marked read-only sections where all information can be viewed.
6	The University of Alabama Homepage http://www.ua.edu
7	UAOPS Homepage
8	Who Uses UAOPS? Regional accrediting organizations such as SACS. Program-specific accrediting bodies such as: ABET for Engineering, AACSB for Business, NCATE for Education, CCNE for Nursing and ABA for Law. Several Others
9	Who Uses UAOPS? UA faculty, staff, and administrators. Provost Office of Institutional Research and Assessment Faculty advisors Other Students, prospective students and their parents. Interested parties such as newspapers and other media and the general public.
10	Why Update Annually? Mandated by SACS Principles of Accreditation
11	Why Update Annually? Mandated by SACS Core Requirement #2.5: “The institution engages in ongoing, integrated, and institution-wide research-based planning and evaluation processes that incorporate a systematic review of programs and services that (a) results in continuing improvement and (b) demonstrates that the institution is effectively accomplishing its mission.”
12	Paradigm Shift of Emphasis Student Learning Outcomes: What students learn Process Orientation: What faculty teach
13	New Emphasis at the Course Level Emphasis at the course level is a new initiative of the accountability movement Course syllabi should contain student learning outcomes that align with those shown at the degree program and department level.
14	Need for Consistency Throughout Organization The University of Alabama Mission College/Dean’s Office Mission Department Mission Degree Program Expected Student Learning Outcomes Course Expected Student Learning Outcomes (Syllabi)
15	Need for Consistency Between Various Documents Throughout Organization The Unit Profile mission statement should be the same as that shown on the departmental website. The UAOPS Unit Profile is actually a component of the Program Review document. The Unit Profile should represent the “backbone” on which more detailed Departmental Strategic Plans are based. There should be congruency between the UAOPS Unit Profile and any assessment plans prepared for the department or program.
16	UAOPS ACADEMIC UNIT PROFILES Each department has a unit profile and each degree program in the department has a unit profile. Example: A department with one undergraduate and two graduate degree programs will have four (4) unit profiles.
17	UAOPS Academic Unit Profile New for 2007/2008: 4-Column Format
18	Also New for 2007/2008 Archived unit profiles from 2005 and 2006 can be viewed by selecting the year. This year, unit profiles will be completed on a prospective basis The outcomes for last year are populated into the grid for 2007/2008, but the remaining areas are blank
19	The New Process for 2007/2008 The 2007/2008 outcomes in column 1 should be edited, changed, or deleted as needed now. The assessment methods planned for those outcomes should be described in column 2 entitled, “Unit Methods of Assessment”. As results are available, they should be added to column 3, entitled, “Unit Results of Assessment”. Please include a data summary. When the assessment results have been reviewed and formally discussed by faculty and management, and curricular changes and improvements have been implemented, that information should be summarized in column 4. The final deadline for completion of academic profiles is May 15, 2008. May 15 was chosen to allow most units to finalize the unit profile along with their annual report. The unit profiles will be evaluated after May 15 and archived on August 20, 2008. A feedback report on the profile will be emailed to the department head and the person who logs in.
20	Feedback to Units When draft unit profiles are completed, they will be evaluated. Feedback reports will be sent to the units again this year. Profiles can be edited until the archive date of August 20, even after the report is submitted.
21	Departmental Expected Outcome Statements (Column 1) Departmental Expected Outcomes tend to be overarching outcomes related to the teaching, research, and service functions of the department’s mission. They are not outcomes specific to individual degree programs.
22	Degree Program Student Learning Outcome Statements (Column 1) Unit Expected Outcomes contain faculty expectations for student learning, i.e., student learning outcomes that are being assessed. Each learning outcome has its own row and it corresponds to the information contained in the same row in columns 2, 3, and 4. Guiding questions should be, “what will students in the major learn and what skills will they develop as a result of this program?” Expectations must be stated in active terms with observable, measurable results. They should be singular in nature, not numerous expectations bundled together. If student learning outcomes are constructed as long sentences punctuated with commas, semi-colons, or conjunctions, they probably contain a number of expectations that each need assessment.
23	Degree Program Student Learning Outcome Statements (Continued) Consider using wording for student learning outcomes such as, “Students will demonstrate _________” or “Students will show evidence of_______”.
24	Student Learning Outcomes Definitions What do faculty believe students should be able to Know, think, and do that they could not do before When they have completed the program of study or course.
25	Examples of Student Learning Outcomes
26	Examples of Action Verbs Appropriate for Student Learning Outcomes For Application Apply Schedule Illustrate Interpret Employ For Evaluation Predict Assess Select Evaluate Estimate For Knowledge Arrange Order Define Recall Relate For Comprehension Classify Locate Report Indicate Review
27	Assessment Methods and Results Columns 2 and 3 of the Unit Profile
28	Assessment Methods and Results (Columns 2 and 3) Methods and data-based results should be summarized for each student learning outcome. Omitting methods and/or results, or failing to include a data summary were the most common error in the 2006/2007 unit profiles. Thresholds for success should be defined and results data should be summarized.
29	Assessment Methods (Column 2) Decide what will be assessed. If there are 8 student learning outcomes and 5 are the most important, assess the 5 most important ones. Don’t try to measure everything. Be patient and flexible. Expect to make changes or improvements based on results. Share the results. Have opportunities for faculty to discuss findings and have feedback loops to annual or strategic planning.
30	Attributes of a Good Assessment Method Each stated student learning outcome should be assessed. Methods and results that include data should be clearly summarized. Multiple measures should be used (triangulate). Assessment method used should “map” to the corresponding student learning outcome. Grades should not be presented as an assessment method. Source: Feedback Sheet for Academic Profiles The University of Alabama
31	Why Not Use Grades? SACS does not accept grades as a measure of student learning outcomes for a degree program. Grading practices are not standard across faculty, courses, and departments. Grades reflect things other than student learning outcomes. Objectives and outcomes differ from course to course and instructor to instructor.
32	How Can “Grades” be Used? --An Example Adapt a rubric example for use with the project or performance Have more than one faculty evaluate performance or project. Discuss, plan and implement curricular changes based on results. Select components that make up the grading process Papers Presentations Other Projects Ensure that assignment measures the learning goals.
33	Assessment Techniques Choose multiple methods for assessment, both direct and indirect. Direct methods: Explicitly and specifically measure the student learning outcome. Indirect methods: Do not directly and specifically measure the student learning outcome. They may ask for opinions or go to secondary sources for data.
34	Examples of Direct Assessment Methods Juried results Internships Standardized Tests Performances Case Studies Team-based or collaborative projects Internships or Service Projects Capstone course results Common examinations (with multiple course sections) Rubrics Student learning portfolios Final student projects Learning artifacts
35	Examples of Indirect Assessment Methods Graduation rates Retention rates Job placement rates Satisfaction/opinion survey data Focus Groups Interviews Professional/graduate admission rates Exam pass rate or average score
36	Rubrics or Rating Scales A rubric is a table that lists specific traits that apply to the skill being taught and defines quality levels for each trait. Useful Websites Kathy Schrock's Guide for Educators - Assessment Rubrics Miami University at Ohio Assessing Critical Thinking (ACT) Project http://www.units.muohio.edu/led/assessment/criticalthinking/index.htm
37	Rubrics Rubrics are used to determine the quality of a student’s performance, often with more than one evaluator. Rubrics are often handed out to students so that they can assess their own performance.
38	Rubrics Rubrics are often used to assess student projects, papers, presentations, etc. Rubrics can make assessment easier with large classes, particularly with group projects. They can be used for student peer review in groups.
39	Improvements Based on Assessment (Column 4 of Unit Profile) Are curricular actions in progress or planned in response to assessment results? Do they directly relate to the corresponding student learning outcome? Is there a process for faculty to discuss assessment findings and improvements? Is there a timeline for implementation of the improvements?
40	Example
41	Example: Master’s in Music Composition Student Learning Outcomes and Assessment Measures Student Learning Outcome: Master’s composition graduates will synthesize the musical language of 20^th Century composers in their work. Possible Sources of Evidence: Student portfolios, course portfolios that include student work such as performances or exams. Possible Ways to Measure: Three faculty raters use a simple rubric designed for this purpose to rate student work collected in student and/or course portfolios. When to Measure: End of first year; master’s performance of their work at end of 2^nd year Possible Standard for Success: The rubric defines levels of mastery of desired attributes. Possible Use of Results: Planning for redesign of assignments, restructure of course sequencing or addition of a course, and allocation of resources for software or new faculty position.
42	Example: Master’s in Music Composition In Old 3-Column Format
43	Example: Master’s in Music Composition In New 4-Column Format (Mission Statement Omitted)
44	The Update Process (In brief)
45	Who Can Update UAOPS? Each unit has a representative or “contact” with a username and password. The contact can update or upload information. Unit Profile updates are entered directly into the text display area.
46	Instructions for Updates are Available on the UAOPS Website Homepage
47	Privacy Considerations UAOPS users who log in are asked to read and agree to a user agreement The agreement describes important privacy rules and makes the login user responsible for insuring that no protected personal information is uploaded into UAOPS.
48	Information on Privacy Rules The University of Alabama website has Family Educational Rights and Privacy Act (FERPA) training presentations available. FERPA 101 is for UA employees. FERPA 102 is for administrators
49	Frequently Asked Questions A list of FAQs relating to use of UAOPS is posted at the UAOPS website. The Institutional Research and Assessment staff are available to help at any time.
50	The Process for 2007/2008 Edit and finalize the current 2007/2008 outcomes in column 1 now. The assessment methods planned for those outcomes should be summarized in column 2 entitled, “Unit Methods of Assessment”. As results are available, they should be added to column 3, entitled, “Unit Results of Assessment”. Include a data summary. When the assessment results have been reviewed and formally discussed by faculty and management, and curricular changes and improvements have been implemented, summarize that information in column 4. The final deadline for completion of academic profiles is May 15, 2008. May 15 was chosen to allow most units to finalize the unit profile along with their annual report. The unit profiles will be evaluated after May 15 and archived on August 20, 2008. A feedback report on the profile will be emailed to the department head and the person who logs in.
51	Annual Archiving of Results Each year, in August, the completed unit profiles are archived to document progress and the University’s commitment to ongoing assessment at the unit level. The feedback reports will be archived along with the unit profile.
52	Questions? This concludes the material on writing unit profiles. A short technical demonstration follows if you are interested. If you do not need the technical demonstration, you are dismissed.
53	Thank you.
54	SOURCES The SACS-COC 2006 Annual Meeting The 2006 Annual Meeting of the Commission on Colleges NPEC Assessment Sourcebook, 2005 “Writing Student Outcomes”, University of Southern California, www.usc.edu, 2006. Walvoord, Barbara E., ASSESSMENT CLEAR AND SIMPLE, Jossey-Bass, 2004.