Oversight: Student learning outcomes are not specific to the program and are more reflective of general education outcomes
Outcomes such as “think critically and analytically,” “speak and write clearly,” and “gather and evaluate data” are all important student outcomes, but they are outcomes that might apply to the goals of the general education curriculum or the institution’s goals for its students. Programs should determine their own specific student learning outcomes that are germane to the discipline. For example, a history program may have a learning outcome that states: students will acquire knowledge about historical processes in a specific region of the world. A biological sciences program might have an outcome that states: students should be able to work collaboratively in a laboratory setting. While a physics program may state that: students should be able to design and set up an experiment, collect and analyze data, interpret results, and connect it to related areas in physics. If one of the most important skills you want your students to learn from your program is how to communicate effectively, it should be framed in a way that reflects the discipline itself. For example, a JD program may have a learning
outcome that states: students will be able to communicate, logically, effectively, and persuasively, in a manner appropriate to audience and purpose, to lawyers and decision makers. The professions, the sciences, and humanities, etc. all have their own style and methods for transmitting knowledge that’s appropriate to the discipline.
Oversight: Learning outcomes outline the requirements for graduation and not what students should know or be able to do after they complete the program.
Student learning outcomes are not intended to be a list of requirements for graduation such as completing courses, writing dissertations, or passing comprehensive examinations. Student learning outcomes are specific skills students should have or activities they should be able to do upon completion of an academic program. Learning outcomes should be reflective of the product rather than the process. Instead of stating that students will write a dissertation as a learning goal, programs might instead state that students who complete their program will be able to successfully engage in original and creative research that has an impact on the discipline. Such a learning outcome might be assessed through tabulating the number of peer-reviewed publications graduates of the program have at specific intervals (by graduation, three years out, five years out, etc.). Or it may be assessed every five years by a team of faculty who, with set criteria in mind, review a sample of dissertations.
Oversight: Courses are used as a method of assessment with no external validation and course grades or GPAs are used as a standard for assessment.
It is not sufficient to state a grade or overall GPA as a measurement of a student learning outcome. A grade is a multifaceted measure of many different activities that occur in a course all collapsed into a single letter or number. Therefore, a letter grade by itself does not give enough information about the learning that was measured or the criteria that were used to do so. Similarly, a GPA is a measurement of many different courses that build a student’s curriculum, but cannot be mapped back into a specific learning outcome.
Many programs choose to use a team of three faculty members who review a sample of student papers from a capstone course and assess how well students did at achieving the specific learning outcomes determined by the program. This method provides the necessary external validation and a clear assessment of specific learning outcomes. Many graduate programs use a similar approach, using a team of faculty to review a percentage of theses and dissertations to assess if graduate students are meeting the standard set by the faculty for specific learning outcomes. This approach need not be time consuming as only a sample of capstone papers, theses, and dissertations would need to be reviewed every three to five years.
Oversight: Assessment methods are not tailored to specific learning goals.
Assessment methods should be strategically selected to best assess the specific learning outcomes. Not all assessment methods are well suited for every learning outcome. For example, the ability to master the core concepts of a profession might be best assessed using a licensure or professional certification exam as opposed to a job placement record. A job placement record might be a good indirect assessment measure if the learning outcomes most important to your major are also important to employers. Graduate school placement might be a good indirect assessment if the most important learning outcomes are those valued by the graduate school. Publications and citations might be a good assessment of graduate students’ ability to conduct independent research. If an assessment plan employs the exact same assessment methods for each learning outcome it would be a good indication
that a closer review of that section is needed.
Oversight: Assessment methods do not contain information on who will be assessed, when they will be assessed and how often assessment will take place.
Once an appropriate method of assessment is identified, the faculty should decide who will be assessed, i.e., what sample of students. In some cases, particularly when reviewing student papers, theses, or dissertations, it makes sense to only use a small sample of students such as 10%. In other cases when, for example, placement records of graduate students are used, 100% of the students might be assessed. When and how often assessment takes place are also important factors to consider. To keep assessment manageable, programs should determine an assessment timetable that results in meaningful data without causing undo burden to the faculty. Student learning outcomes need only be assessed every three to five years. A schedule that takes the entire plan into consideration can and should result in only a small time commitment on the part of the program’s faculty or staff each year.
Oversight: Standards do not identify the percentage of students who should achieve the stated outcome.
In order to assess progress, faculty should determine what percentage of students should meet each learning outcome. Though it may be difficult to determine a percentage in the beginning, programs can adjust their standards as they receive the results of the assessment and get a clearer idea of how their students are achieving the goals set by the program. In some programs, it may be unrealistic to set a standard that 100% of students can meet, for example, having a publication in print by graduation. While in other cases, it may be reasonable for 100% of students to meet a given standard, such as successfully applying the theories and methods of the given field in a capstone paper.