This supplement to the Guide to How to Assess Student Learning at the Program Level provides samples of documents departments and programs need in order to conduct program-level assessment of student learning effectively as part of an
assessment plan
Assessment Plan A plan for assessing student learning at the program level that includes:
defined program-level student learning outcomes a curriculum map that describes where in the curriculum each outcome is assessed a reasonable timetable that determines when and where in the curriculum each outcome is assessed periodic reflection on the results and the process to inform curricular and programmatic planning and decisions program-level student learning outcomes
Program-Level Student Learning Outcomes Statements describing what your students should be able to do as a result of completing your degree program. Outcomes emphasize the integration and application of knowledge rather than coverage of material, and are observable, measurable, and demonstrable. They use specific, active verbs (e.g., “identify,” “develop,” “communicate,” “demonstrate”) rather than “understand.” Program-level student learning outcomes are often abbreviated as program-level SLOs or as PLOs , and are also known as program-level learning goals . The term “outcomes” is becoming preferred over “goals” or “objectives” because it makes it clearer that these are defined expectations upon completion of the program, rather than aspirational goals that may or may not be achieved. Examples include:
Identify, formulate, and solve broadly defined technical or scientific problems by applying knowledge of mathematics and science and/or technical topics to areas relevant to the discipline Develop and conduct experiments or test hypotheses, analyze and interpret data, and use scientific judgment to draw conclusions Communicate scientific ideas and results in written and oral form according to professional standards and norms Demonstrate and exemplify an understanding of ethical conduct in scientific and professional settings Program-level student learning outcomes generally focus on overall program outcomes, in contrast to course-level student learning outcomes, which are specific to the knowledge and skills addressed in individual courses. Accreditation requirements typically require program-level student learning outcomes to be defined separately for each degree program (e.g., BA, BS, or minor), even though there will often be considerable overlap among these sets of outcomes. For more details, see the section on How to Assess Student Learning at the Program Level . For examples, see the supplement on Sample Documents for Program-Level Assessment of Student Learning or the PhysPort expert recommendation How do I develop student learning outcomes for physics courses?
rubrics
Rubric A tool that identifies criteria for performance and different levels of performance within each of those criteria. A rubric may be used to articulate the expectations of student achievement for an assignment or key performance indicator (KPI) as defined by instructional staff or by a department. Rubrics provide a consistent, impartial evaluation of an assignment or performance indicator by defining explicit, developmentally appropriate criteria for different levels of performance. Rubrics come in many forms, from a checklist of required items to a detailed matrix with succinct, explicit descriptions of performance levels for each criteria. See the section on How to Select and Use Various Assessment Methods in your Program for details on how to use departmental rubrics and rubrics of student performance . For examples of rubrics for program-level student learning outcomes, see the supplement on Sample Documents for Program-Level Assessment of Student Learning.
curriculum maps
Curriculum Map A document used for curricular design that identifies the learning opportunities designed to address each program-level student learning outcome. An assessment plan typically includes a simplified version of a curriculum map, which shows only when and where in the curriculum (typically in courses) each program-level student learning outcome is assessed. Program-level student learning outcomes may be addressed and/or assessed in curricular and co-curricular activities, including courses, laboratory experiences, seminars, internships, research, and capstone experiences. Robust curricular design and student learning assessment plans don’t merely address or assess an outcome at one point in the curriculum; rather, each outcome is addressed at different levels (introduction, emphasis, reinforcement) throughout the curriculum and assessed in several places and at different degrees of mastery (emerging, developing, proficient). Consult your office of assessment for institution-specific recommendations. For more details, see the section on How to Assess Student Learning at the Program Level .
These samples are intended to provide models of assessment documents that you can use as starting points or guides for developing documents that are appropriate to the needs of your faculty and program. By design, the sample documents are not complete. These samples are intended to illustrate the conceptual structure of these documents, not to provide ready-to-use templates. For assessment of student learning to be effectively used to improve your department’s program, it should be aligned with your department’s mission and vision (see How to Create and Use Foundational Documents ) and tailored to your department’s curriculum and local conditions.
The samples are also, by design, simple versions of documents a department might develop, in keeping with the principles discussed in How to Assess Student Learning at the Program Level in the guidance on how to keep your assessment plan manageable .
Because
accreditation
Accreditation A “process of self-review and peer review for improvement of academic quality and public accountability of institutions and programs. This quality review process occurs on a period basis, usually every three to ten years.” (Definition from CHEA .) The EP3 Initiative takes the position that physics programs should use the program-level assessment of student learning required for accreditation as an opportunity to engage in a cyclic process of program improvement and to design learning assessments that support goals for improving your program while satisfying accreditation requirements, avoiding duplicating efforts or doing busywork that doesn’t support your department. Nearly every college or university in the U.S. is accredited by one of the seven major accreditation agencies for four-year colleges and universities , and you need to know which one is your accreditor. Requirements vary among agencies, but all are concerned with learning assessment. The Council for Higher Education Accreditation (CHEA) is an association of degree-granting colleges and universities in the US that recognizes institutional and programmatic accrediting organizations. The Council of Regional Accrediting Commissions (C-RAC) is an association of accreditation agencies. Physics programs also need to ensure that physics courses for engineering and chemistry majors meet, respectively, the requirements of ABET , the accrediting body for most engineering programs in the U.S. (including some engineering physics and physics programs), and the American Chemical Society (ACS) , which approves chemistry degree programs in the US.
course-
Course-Level Student Learning Outcomes Statements describing what students should be able to do as a result of completing a particular course. Outcomes emphasize the integration and application of knowledge rather than coverage of material, and are observable, measurable, and demonstrable. They use specific, active verbs (e.g., “solve,” “describe,” and “calculate”) rather than “understand.” Course-level student learning outcomes are often abbreviated as course-level SLOs and are also known as course-level learning goals . Examples include:
Solve the Schroedinger equation in one dimension for commonly encountered simple potentials Describe physical situations that correspond to simple potential energy curves Calculate the electric field or potential due to a system of charges using Coulomb’s law Course-level student learning outcomes are generally specific to the knowledge and skills addressed in individual courses, in contrast to program-level student learning outcomes, which focus on overall program outcomes. For instructional staff, these learning outcomes clarify what the course will deliver and unite course content with course-level assessments. Specifying course-level learning outcomes in individual course syllabi is often a requirement for accreditation of your institution, or of the institution itself. Assessment of course-level student learning outcomes through course assignments or examinations should be aligned with assessment of program-level learning outcomes, when possible. See the section on Implementing Research-Based Instructional Practices for guidance on how to design and assess courses based on program-level and course-level student learning outcomes. For examples, see the PhysPort expert recommendation How do I develop student learning outcomes for physics courses?
program-level student learning outcomes
Program-Level Student Learning Outcomes Statements describing what your students should be able to do as a result of completing your degree program. Outcomes emphasize the integration and application of knowledge rather than coverage of material, and are observable, measurable, and demonstrable. They use specific, active verbs (e.g., “identify,” “develop,” “communicate,” “demonstrate”) rather than “understand.” Program-level student learning outcomes are often abbreviated as program-level SLOs or as PLOs , and are also known as program-level learning goals . The term “outcomes” is becoming preferred over “goals” or “objectives” because it makes it clearer that these are defined expectations upon completion of the program, rather than aspirational goals that may or may not be achieved. Examples include:
Identify, formulate, and solve broadly defined technical or scientific problems by applying knowledge of mathematics and science and/or technical topics to areas relevant to the discipline Develop and conduct experiments or test hypotheses, analyze and interpret data, and use scientific judgment to draw conclusions Communicate scientific ideas and results in written and oral form according to professional standards and norms Demonstrate and exemplify an understanding of ethical conduct in scientific and professional settings Program-level student learning outcomes generally focus on overall program outcomes, in contrast to course-level student learning outcomes, which are specific to the knowledge and skills addressed in individual courses. Accreditation requirements typically require program-level student learning outcomes to be defined separately for each degree program (e.g., BA, BS, or minor), even though there will often be considerable overlap among these sets of outcomes. For more details, see the section on How to Assess Student Learning at the Program Level . For examples, see the supplement on Sample Documents for Program-Level Assessment of Student Learning or the PhysPort expert recommendation How do I develop student learning outcomes for physics courses?
How do I develop student learning outcomes for physics courses? Sample Program-Level Student Learning Outcomes
Program-level student learning outcomes
Program-Level Student Learning Outcomes Statements describing what your students should be able to do as a result of completing your degree program. Outcomes emphasize the integration and application of knowledge rather than coverage of material, and are observable, measurable, and demonstrable. They use specific, active verbs (e.g., “identify,” “develop,” “communicate,” “demonstrate”) rather than “understand.” Program-level student learning outcomes are often abbreviated as program-level SLOs or as PLOs , and are also known as program-level learning goals . The term “outcomes” is becoming preferred over “goals” or “objectives” because it makes it clearer that these are defined expectations upon completion of the program, rather than aspirational goals that may or may not be achieved. Examples include:
Identify, formulate, and solve broadly defined technical or scientific problems by applying knowledge of mathematics and science and/or technical topics to areas relevant to the discipline Develop and conduct experiments or test hypotheses, analyze and interpret data, and use scientific judgment to draw conclusions Communicate scientific ideas and results in written and oral form according to professional standards and norms Demonstrate and exemplify an understanding of ethical conduct in scientific and professional settings Program-level student learning outcomes generally focus on overall program outcomes, in contrast to course-level student learning outcomes, which are specific to the knowledge and skills addressed in individual courses. Accreditation requirements typically require program-level student learning outcomes to be defined separately for each degree program (e.g., BA, BS, or minor), even though there will often be considerable overlap among these sets of outcomes. For more details, see the section on How to Assess Student Learning at the Program Level . For examples, see the supplement on Sample Documents for Program-Level Assessment of Student Learning or the PhysPort expert recommendation How do I develop student learning outcomes for physics courses?
How to Assess Student Learning at the Program Level for guidance on how to develop program-level student learning outcomes for each degree program . The statement of each outcome may be accompanied by a list of the key performance indicators (KPIs)
Key Performance Indicator (KPI) Measurable indicators of performance for a specific objective. KPIs are often numerical and tracked over time. Also sometimes referred to as performance indicators or measurable outcomes .
Identifying and assessing KPIs can help drive departmental improvement and identify successes by tracking performance over time. Examples of KPIs that provide insight into how well your department is performing include courses offered, student credit hours taught, and students enrolled. Institutions sometimes require that KPIs be measured against predetermined benchmarks, e.g., “At least 80% of students will…”.
KPIs may also be used to measure overall student performance when assessing student learning at the program level. This requires developing KPIs that define what it means to attain each program-level student learning outcome. For example, for the program-level student learning outcome, “demonstrate competency in effective communication skills to a range of audiences,” two performance indicators could be, “prepare and deliver an effective oral presentation” and “construct a well-written paper.” Student achievement for each performance indicator is assessed using a developmentally appropriate rubric. The aggregated expected achievement level for first-year students would be lower than the expected level for seniors. For examples of KPIs for program-level student learning outcomes, see the supplement on Sample Documents for Program-Level Assessment of Student Learning.
rubric
Rubric A tool that identifies criteria for performance and different levels of performance within each of those criteria. A rubric may be used to articulate the expectations of student achievement for an assignment or key performance indicator (KPI) as defined by instructional staff or by a department. Rubrics provide a consistent, impartial evaluation of an assignment or performance indicator by defining explicit, developmentally appropriate criteria for different levels of performance. Rubrics come in many forms, from a checklist of required items to a detailed matrix with succinct, explicit descriptions of performance levels for each criteria. See the section on How to Select and Use Various Assessment Methods in your Program for details on how to use departmental rubrics and rubrics of student performance . For examples of rubrics for program-level student learning outcomes, see the supplement on Sample Documents for Program-Level Assessment of Student Learning.
In physics, program-level student learning outcomes commonly tend to be organized around a set of broad themes, such as content knowledge, problem-solving skills, sense-making or analytical skills, experimental skills, computational skills, independent inquiry and research skills, scientific communication, teamwork, and the ethical conduct of science.
For example, program-level student learning outcomes about problem-solving and scientific communication skills might read, in part:
Sample Outcome 1: Problem Solving Physics majors and minors will demonstrate the ability to solve complex physics problems.
Key performance indicators (KPIs)
Key Performance Indicator (KPI) Measurable indicators of performance for a specific objective. KPIs are often numerical and tracked over time. Also sometimes referred to as performance indicators or measurable outcomes .
Identifying and assessing KPIs can help drive departmental improvement and identify successes by tracking performance over time. Examples of KPIs that provide insight into how well your department is performing include courses offered, student credit hours taught, and students enrolled. Institutions sometimes require that KPIs be measured against predetermined benchmarks, e.g., “At least 80% of students will…”.
KPIs may also be used to measure overall student performance when assessing student learning at the program level. This requires developing KPIs that define what it means to attain each program-level student learning outcome. For example, for the program-level student learning outcome, “demonstrate competency in effective communication skills to a range of audiences,” two performance indicators could be, “prepare and deliver an effective oral presentation” and “construct a well-written paper.” Student achievement for each performance indicator is assessed using a developmentally appropriate rubric. The aggregated expected achievement level for first-year students would be lower than the expected level for seniors. For examples of KPIs for program-level student learning outcomes, see the supplement on Sample Documents for Program-Level Assessment of Student Learning.
Identify relevant information and decide on a strategy for solving a problem. Determine and articulate appropriate assumptions necessary to reach a solution. Create a mathematical model of the physical system using these assumptions, employing, e.g., geometry, algebraic relationships, and differential equations. Represent the elements of a problem and its solution in multiple ways (e.g., with words or sketches, graphical representations, and mathematical models) and explain and use the relationships between these representations to solve the problem and interpret its solution. Assess the physical viability of a proposed solution. … Sample Outcome 2: Scientific Communication Physics majors will demonstrate the ability to clearly communicate scientific concepts and information.
Key performance indicators (KPIs)
Key Performance Indicator (KPI) Measurable indicators of performance for a specific objective. KPIs are often numerical and tracked over time. Also sometimes referred to as performance indicators or measurable outcomes .
Identifying and assessing KPIs can help drive departmental improvement and identify successes by tracking performance over time. Examples of KPIs that provide insight into how well your department is performing include courses offered, student credit hours taught, and students enrolled. Institutions sometimes require that KPIs be measured against predetermined benchmarks, e.g., “At least 80% of students will…”.
KPIs may also be used to measure overall student performance when assessing student learning at the program level. This requires developing KPIs that define what it means to attain each program-level student learning outcome. For example, for the program-level student learning outcome, “demonstrate competency in effective communication skills to a range of audiences,” two performance indicators could be, “prepare and deliver an effective oral presentation” and “construct a well-written paper.” Student achievement for each performance indicator is assessed using a developmentally appropriate rubric. The aggregated expected achievement level for first-year students would be lower than the expected level for seniors. For examples of KPIs for program-level student learning outcomes, see the supplement on Sample Documents for Program-Level Assessment of Student Learning.
Present physical concepts, mathematical reasoning, and results of experiments through clear and well-organized scientific and/or technical written reports that are consistent with professional norms. Present scientific concepts and results through clear and well-organized oral presentations. Create clear and effective diagrams and graphs to communicate scientific concepts and results. … Sample Curriculum Maps Your program’s
curriculum map
Curriculum Map A document used for curricular design that identifies the learning opportunities designed to address each program-level student learning outcome. An assessment plan typically includes a simplified version of a curriculum map, which shows only when and where in the curriculum (typically in courses) each program-level student learning outcome is assessed. Program-level student learning outcomes may be addressed and/or assessed in curricular and co-curricular activities, including courses, laboratory experiences, seminars, internships, research, and capstone experiences. Robust curricular design and student learning assessment plans don’t merely address or assess an outcome at one point in the curriculum; rather, each outcome is addressed at different levels (introduction, emphasis, reinforcement) throughout the curriculum and assessed in several places and at different degrees of mastery (emerging, developing, proficient). Consult your office of assessment for institution-specific recommendations. For more details, see the section on How to Assess Student Learning at the Program Level .
program-level student learning outcome
Program-Level Student Learning Outcomes Statements describing what your students should be able to do as a result of completing your degree program. Outcomes emphasize the integration and application of knowledge rather than coverage of material, and are observable, measurable, and demonstrable. They use specific, active verbs (e.g., “identify,” “develop,” “communicate,” “demonstrate”) rather than “understand.” Program-level student learning outcomes are often abbreviated as program-level SLOs or as PLOs , and are also known as program-level learning goals . The term “outcomes” is becoming preferred over “goals” or “objectives” because it makes it clearer that these are defined expectations upon completion of the program, rather than aspirational goals that may or may not be achieved. Examples include:
Identify, formulate, and solve broadly defined technical or scientific problems by applying knowledge of mathematics and science and/or technical topics to areas relevant to the discipline Develop and conduct experiments or test hypotheses, analyze and interpret data, and use scientific judgment to draw conclusions Communicate scientific ideas and results in written and oral form according to professional standards and norms Demonstrate and exemplify an understanding of ethical conduct in scientific and professional settings Program-level student learning outcomes generally focus on overall program outcomes, in contrast to course-level student learning outcomes, which are specific to the knowledge and skills addressed in individual courses. Accreditation requirements typically require program-level student learning outcomes to be defined separately for each degree program (e.g., BA, BS, or minor), even though there will often be considerable overlap among these sets of outcomes. For more details, see the section on How to Assess Student Learning at the Program Level . For examples, see the supplement on Sample Documents for Program-Level Assessment of Student Learning or the PhysPort expert recommendation How do I develop student learning outcomes for physics courses?
How to Assess Student Learning at the Program Level for guidance on how to create a curriculum map and rubrics . Each outcome is addressed at different levels (e.g., “introduce,” “practice,” and “reinforce”) in the curriculum and assessed in several places and at different developmentally appropriate degrees of mastery (e.g., “emerging,” “developing,” and “proficient”).For purposes of program-level assessment of student learning, a curriculum map is a guide to where in your curriculum each program-level learning outcome is assessed. It does not need to identify every course in which aspects of each learning outcome may be addressed. Indeed, while it is expected that many program-level student learning outcomes will be at least touched upon in most courses, avoid assessing more than two outcomes in any given course (one is preferable, if possible) to avoid overburdening faculty with assessment responsibilities. Instead, identify three to five courses in which to assess each program-level student learning outcome longitudinally through the curriculum, to help your program understand how student mastery of that outcome progresses through the curriculum.
Curriculum maps may also be used for other purposes, such as “curricular alignment ”, which involves designing a curriculum around program-level student learning outcomes. In such cases, a curriculum map may specify every place in the curriculum that each program-level learning outcome and each
key performance indicator (KPI)
Key Performance Indicator (KPI) Measurable indicators of performance for a specific objective. KPIs are often numerical and tracked over time. Also sometimes referred to as performance indicators or measurable outcomes .
Identifying and assessing KPIs can help drive departmental improvement and identify successes by tracking performance over time. Examples of KPIs that provide insight into how well your department is performing include courses offered, student credit hours taught, and students enrolled. Institutions sometimes require that KPIs be measured against predetermined benchmarks, e.g., “At least 80% of students will…”.
KPIs may also be used to measure overall student performance when assessing student learning at the program level. This requires developing KPIs that define what it means to attain each program-level student learning outcome. For example, for the program-level student learning outcome, “demonstrate competency in effective communication skills to a range of audiences,” two performance indicators could be, “prepare and deliver an effective oral presentation” and “construct a well-written paper.” Student achievement for each performance indicator is assessed using a developmentally appropriate rubric. The aggregated expected achievement level for first-year students would be lower than the expected level for seniors. For examples of KPIs for program-level student learning outcomes, see the supplement on Sample Documents for Program-Level Assessment of Student Learning.
Many institutions will provide a template curriculum map for your programs that your department may be required to use, or have specific requirements concerning your curriculum map. Familiarize yourself with these requirements.
These samples assume a program that has four program-level student learning outcomes:
Problem Solving Experimental and Computational Skills Scientific Communication Research Skills These samples also assume a fairly standard undergraduate curriculum consisting of an introductory physics sequence (including lab with lab reports); sophomore-level courses in computational physics and modern physics (including lab with lab reports); junior- and senior-level courses in analytical mechanics, electromagnetism, quantum mechanics, statistical physics, and mathematical methods; an advanced lab course; and a senior capstone course that involves a research project and a written capstone. The sample curriculum maps shown do make specific assumptions about the content and structure of these courses that are not explicitly stated, e.g., that the computational physics course incorporates an independent research project.
Download all sample curriculum maps as a spreadsheet
Sample Curriculum Map One A minimalist example that simply identifies in which courses each program-level student learning outcome is assessed. A simple curriculum map like this is a good place for a program to get started with program-level learning assessment. The program can develop a more sophisticated curriculum map as it gains experience with what works best for its faculty in its local context.
Sample Curriculum Map Two The same curriculum map as the previous one, but indicating a developmentally appropriate degree of mastery (e.g., “emerging,” “developing,” and “proficient”) relative to which the outcome will be assessed at that point in the curriculum.
Sample Curriculum Map Three The same curriculum map as the previous one, but breaking down which
key performance indicators (KPIs)
Key Performance Indicator (KPI) Measurable indicators of performance for a specific objective. KPIs are often numerical and tracked over time. Also sometimes referred to as performance indicators or measurable outcomes .
Identifying and assessing KPIs can help drive departmental improvement and identify successes by tracking performance over time. Examples of KPIs that provide insight into how well your department is performing include courses offered, student credit hours taught, and students enrolled. Institutions sometimes require that KPIs be measured against predetermined benchmarks, e.g., “At least 80% of students will…”.
KPIs may also be used to measure overall student performance when assessing student learning at the program level. This requires developing KPIs that define what it means to attain each program-level student learning outcome. For example, for the program-level student learning outcome, “demonstrate competency in effective communication skills to a range of audiences,” two performance indicators could be, “prepare and deliver an effective oral presentation” and “construct a well-written paper.” Student achievement for each performance indicator is assessed using a developmentally appropriate rubric. The aggregated expected achievement level for first-year students would be lower than the expected level for seniors. For examples of KPIs for program-level student learning outcomes, see the supplement on Sample Documents for Program-Level Assessment of Student Learning.
instructional staff
Instructional Staff Faculty, instructors, adjuncts, teaching staff, and others who serve as instructors of record for courses. This term does not include instructional support staff who support the teaching of courses.
A more elaborate version of such a map might also list the specific course elements (e.g., designated homework or exam questions, projects, presentations) used to assess each KPI.
Sample Curriculum Map Four The same curriculum map as the previous one, but identifying every place in the curriculum where each program-level learning outcome and associated
key performance indicator (KPI)
Key Performance Indicator (KPI) Measurable indicators of performance for a specific objective. KPIs are often numerical and tracked over time. Also sometimes referred to as performance indicators or measurable outcomes .
Identifying and assessing KPIs can help drive departmental improvement and identify successes by tracking performance over time. Examples of KPIs that provide insight into how well your department is performing include courses offered, student credit hours taught, and students enrolled. Institutions sometimes require that KPIs be measured against predetermined benchmarks, e.g., “At least 80% of students will…”.
KPIs may also be used to measure overall student performance when assessing student learning at the program level. This requires developing KPIs that define what it means to attain each program-level student learning outcome. For example, for the program-level student learning outcome, “demonstrate competency in effective communication skills to a range of audiences,” two performance indicators could be, “prepare and deliver an effective oral presentation” and “construct a well-written paper.” Student achievement for each performance indicator is assessed using a developmentally appropriate rubric. The aggregated expected achievement level for first-year students would be lower than the expected level for seniors. For examples of KPIs for program-level student learning outcomes, see the supplement on Sample Documents for Program-Level Assessment of Student Learning.
assessed , along with the expected level of mastery (e.g., “emerging,” “developing,” and “proficient”) relative to which that KPI will be assessed. A curriculum map of this type is used in the process of curricular alignment , which involves designing a curriculum around program-level student learning outcomes, and contains much more detail than is necessary for learning assessment alone.A more elaborate version of such a map might also list the specific learning opportunities designed to support each KPI addressed in a course.
Sample Rubrics and Reporting Forms for Program-Level Student Learning Outcomes Each
program-level student learning outcome
Program-Level Student Learning Outcomes Statements describing what your students should be able to do as a result of completing your degree program. Outcomes emphasize the integration and application of knowledge rather than coverage of material, and are observable, measurable, and demonstrable. They use specific, active verbs (e.g., “identify,” “develop,” “communicate,” “demonstrate”) rather than “understand.” Program-level student learning outcomes are often abbreviated as program-level SLOs or as PLOs , and are also known as program-level learning goals . The term “outcomes” is becoming preferred over “goals” or “objectives” because it makes it clearer that these are defined expectations upon completion of the program, rather than aspirational goals that may or may not be achieved. Examples include:
Identify, formulate, and solve broadly defined technical or scientific problems by applying knowledge of mathematics and science and/or technical topics to areas relevant to the discipline Develop and conduct experiments or test hypotheses, analyze and interpret data, and use scientific judgment to draw conclusions Communicate scientific ideas and results in written and oral form according to professional standards and norms Demonstrate and exemplify an understanding of ethical conduct in scientific and professional settings Program-level student learning outcomes generally focus on overall program outcomes, in contrast to course-level student learning outcomes, which are specific to the knowledge and skills addressed in individual courses. Accreditation requirements typically require program-level student learning outcomes to be defined separately for each degree program (e.g., BA, BS, or minor), even though there will often be considerable overlap among these sets of outcomes. For more details, see the section on How to Assess Student Learning at the Program Level . For examples, see the supplement on Sample Documents for Program-Level Assessment of Student Learning or the PhysPort expert recommendation How do I develop student learning outcomes for physics courses?
key performance indicators (KPIs)
Key Performance Indicator (KPI) Measurable indicators of performance for a specific objective. KPIs are often numerical and tracked over time. Also sometimes referred to as performance indicators or measurable outcomes .
Identifying and assessing KPIs can help drive departmental improvement and identify successes by tracking performance over time. Examples of KPIs that provide insight into how well your department is performing include courses offered, student credit hours taught, and students enrolled. Institutions sometimes require that KPIs be measured against predetermined benchmarks, e.g., “At least 80% of students will…”.
KPIs may also be used to measure overall student performance when assessing student learning at the program level. This requires developing KPIs that define what it means to attain each program-level student learning outcome. For example, for the program-level student learning outcome, “demonstrate competency in effective communication skills to a range of audiences,” two performance indicators could be, “prepare and deliver an effective oral presentation” and “construct a well-written paper.” Student achievement for each performance indicator is assessed using a developmentally appropriate rubric. The aggregated expected achievement level for first-year students would be lower than the expected level for seniors. For examples of KPIs for program-level student learning outcomes, see the supplement on Sample Documents for Program-Level Assessment of Student Learning.
rubric
Rubric A tool that identifies criteria for performance and different levels of performance within each of those criteria. A rubric may be used to articulate the expectations of student achievement for an assignment or key performance indicator (KPI) as defined by instructional staff or by a department. Rubrics provide a consistent, impartial evaluation of an assignment or performance indicator by defining explicit, developmentally appropriate criteria for different levels of performance. Rubrics come in many forms, from a checklist of required items to a detailed matrix with succinct, explicit descriptions of performance levels for each criteria. See the section on How to Select and Use Various Assessment Methods in your Program for details on how to use departmental rubrics and rubrics of student performance . For examples of rubrics for program-level student learning outcomes, see the supplement on Sample Documents for Program-Level Assessment of Student Learning.
How to Assess Student Learning at the Program Level for guidance on how to create a curriculum map and rubrics . Such rubrics may also serve as a convenient way to report the results of assessment of an outcome via specific curricular elements according to your program’s curriculum map
Curriculum Map A document used for curricular design that identifies the learning opportunities designed to address each program-level student learning outcome. An assessment plan typically includes a simplified version of a curriculum map, which shows only when and where in the curriculum (typically in courses) each program-level student learning outcome is assessed. Program-level student learning outcomes may be addressed and/or assessed in curricular and co-curricular activities, including courses, laboratory experiences, seminars, internships, research, and capstone experiences. Robust curricular design and student learning assessment plans don’t merely address or assess an outcome at one point in the curriculum; rather, each outcome is addressed at different levels (introduction, emphasis, reinforcement) throughout the curriculum and assessed in several places and at different degrees of mastery (emerging, developing, proficient). Consult your office of assessment for institution-specific recommendations. For more details, see the section on How to Assess Student Learning at the Program Level .
Many institutions will provide a template for reporting assessment results that your department may be required to use, or have specific requirements concerning how you report assessment results. Familiarize yourself with these requirements.
For each outcome assessed, use a rubric to evaluate student attainment of some or all of the KPIs associated with that outcome, using a handful of designated assignments or other artifacts of student work chosen as
assessment measures
Assessment Measures Student assignments or tasks used to assess program-level or course-level student learning outcomes by gathering information about aggregate student performance in your program. Examples include presentations; projects; and designated in-class conceptual questions, homework, and exam questions chosen for their relevance to the particular outcome being assessed. Assessment measures are designed to help you understand how well your program is achieving its overall objectives, with information about performance of individual students redacted from the final results. There are two main types of assessment measures: direct and indirect. Direct assessment measures provide for the direct examination or observation of student knowledge or skills relative to specific learning outcomes. Examples of direct measures include homework problems, exam questions, presentations, and projects. Indirect assessment measures are not tied to specific learning outcomes and should not be used to assess these outcomes, but can provide more nuanced information about overall student performance, success, and/or self-perception of learning. Examples of indirect measures include faculty reports of student learning, records of student success such as post-graduation employment, and grades.
Artifacts of student work based on these measures are evaluated (often against a rubric) to determine how well your program is meeting its objectives. Grading and learning assessment are related but distinct. Grading measures student performance against a rubric that evaluates overall performance, whereas learning assessment uses a rubric tied to attainment of a specific learning outcome. For example, a rubric for grading a lab report might include factors such as correct use of physics for analysis, units, and error analysis. A standardized rubric for assessing the learning outcome of written communication, by contrast, would assess exclusively the written component of the same lab report. Performance relative to this rubric is not used to evaluate individual student achievement, but is instead analyzed in aggregate to evaluate how well the program is helping students attain the assessed outcome and what changes could improve the program.
instructional staff
Instructional Staff Faculty, instructors, adjuncts, teaching staff, and others who serve as instructors of record for courses. This term does not include instructional support staff who support the teaching of courses.
How to Assess Student Learning at the Program Level for additional guidance on how to conduct assessment of program-level student learning outcomes .It is not necessary to assess all of the indicators for an outcome for every assessment measure, or even to assess all of the indicators for an outcome in the same course.
These sample rubrics for assessing program-level student learning outcomes might be used to help instructional staff assess developmentally appropriate mastery of performance indicators associated with a particular outcome. The instructor of each course identified in your program’s curriculum map for this outcome would use the appropriate rubric to assess the student work chosen as the assessment measure for that outcome in the course and record and report the results. The full set of rubrics used to assess this outcome in a given assessment cycle would then be collected, reported, and used as the basis for making programmatic and curricular decisions. See the section on How to Assess Student Learning at the Program Level for additional guidance on how to use assessment results to inform programmatic change (“close the loop”) .
These sample rubrics use the sample learning outcomes “Problem Solving” and/or “Scientific Communication” described above.
Download all sample rubrics and reporting forms as a Word document
An instructor could use this sample rubric to evaluate a particular direct
assessment measure
Assessment Measures Student assignments or tasks used to assess program-level or course-level student learning outcomes by gathering information about aggregate student performance in your program. Examples include presentations; projects; and designated in-class conceptual questions, homework, and exam questions chosen for their relevance to the particular outcome being assessed. Assessment measures are designed to help you understand how well your program is achieving its overall objectives, with information about performance of individual students redacted from the final results. There are two main types of assessment measures: direct and indirect. Direct assessment measures provide for the direct examination or observation of student knowledge or skills relative to specific learning outcomes. Examples of direct measures include homework problems, exam questions, presentations, and projects. Indirect assessment measures are not tied to specific learning outcomes and should not be used to assess these outcomes, but can provide more nuanced information about overall student performance, success, and/or self-perception of learning. Examples of indirect measures include faculty reports of student learning, records of student success such as post-graduation employment, and grades.
Artifacts of student work based on these measures are evaluated (often against a rubric) to determine how well your program is meeting its objectives. Grading and learning assessment are related but distinct. Grading measures student performance against a rubric that evaluates overall performance, whereas learning assessment uses a rubric tied to attainment of a specific learning outcome. For example, a rubric for grading a lab report might include factors such as correct use of physics for analysis, units, and error analysis. A standardized rubric for assessing the learning outcome of written communication, by contrast, would assess exclusively the written component of the same lab report. Performance relative to this rubric is not used to evaluate individual student achievement, but is instead analyzed in aggregate to evaluate how well the program is helping students attain the assessed outcome and what changes could improve the program.
A simple rubric like this is a good place for a program to get started with program-level learning assessment. The program can develop more sophisticated rubrics as it gains experience with what works best for its instructional staff in its local context.
Sample Outcome Rubric Two This rubric provides a partial example of a rubric that elaborates on the previous example by defining specific criteria for evaluating each key performance indicator related to the outcome it is used to assess.
Sample Outcome Rubric Three This combined outcome rubric and reporting form provides four performance levels to evaluate artifacts of student work, report results and observations, make recommendations for specific programmatic or curricular actions, and comment on the impact of previous curricular changes related to the outcome. In the spirit of keeping the assessment process as simple as possible for faculty, it does not provide detailed criteria for each performance indicator.
This example is inspired by rubrics that have been used at Le Moyne College and Oregon State University.
See How to Select and Use Various Assessment Methods in Your Program for information about how to design and use rubrics effectively.
Sample Assessment Timetables These samples provide five different examples of timetables that each ensure that each
program-level student learning outcome
Program-Level Student Learning Outcomes Statements describing what your students should be able to do as a result of completing your degree program. Outcomes emphasize the integration and application of knowledge rather than coverage of material, and are observable, measurable, and demonstrable. They use specific, active verbs (e.g., “identify,” “develop,” “communicate,” “demonstrate”) rather than “understand.” Program-level student learning outcomes are often abbreviated as program-level SLOs or as PLOs , and are also known as program-level learning goals . The term “outcomes” is becoming preferred over “goals” or “objectives” because it makes it clearer that these are defined expectations upon completion of the program, rather than aspirational goals that may or may not be achieved. Examples include:
Identify, formulate, and solve broadly defined technical or scientific problems by applying knowledge of mathematics and science and/or technical topics to areas relevant to the discipline Develop and conduct experiments or test hypotheses, analyze and interpret data, and use scientific judgment to draw conclusions Communicate scientific ideas and results in written and oral form according to professional standards and norms Demonstrate and exemplify an understanding of ethical conduct in scientific and professional settings Program-level student learning outcomes generally focus on overall program outcomes, in contrast to course-level student learning outcomes, which are specific to the knowledge and skills addressed in individual courses. Accreditation requirements typically require program-level student learning outcomes to be defined separately for each degree program (e.g., BA, BS, or minor), even though there will often be considerable overlap among these sets of outcomes. For more details, see the section on How to Assess Student Learning at the Program Level . For examples, see the supplement on Sample Documents for Program-Level Assessment of Student Learning or the PhysPort expert recommendation How do I develop student learning outcomes for physics courses?
curriculum map
Curriculum Map A document used for curricular design that identifies the learning opportunities designed to address each program-level student learning outcome. An assessment plan typically includes a simplified version of a curriculum map, which shows only when and where in the curriculum (typically in courses) each program-level student learning outcome is assessed. Program-level student learning outcomes may be addressed and/or assessed in curricular and co-curricular activities, including courses, laboratory experiences, seminars, internships, research, and capstone experiences. Robust curricular design and student learning assessment plans don’t merely address or assess an outcome at one point in the curriculum; rather, each outcome is addressed at different levels (introduction, emphasis, reinforcement) throughout the curriculum and assessed in several places and at different degrees of mastery (emerging, developing, proficient). Consult your office of assessment for institution-specific recommendations. For more details, see the section on How to Assess Student Learning at the Program Level .
How to Assess Student Learning at the Program Level for guidance on how to establish a timetable for assessment of your program learning outcomes . Ideally, such a timetable should include opportunities to revisit outcomes for which previous assessment results suggested programmatic or curricular changes in order to help your program close the loop
Closing the Loop Using meaningful, effective assessment to improve a program through a continual, cyclic process embedded in the program’s practices and culture, rather than as an occasional activity that serves only to produce a required report. Such a process includes analyzing and reflecting on the results of assessments of program-level student learning outcomes, changing the curriculum to improve student learning, and subsequently re-assessing outcomes to determine the effect of your changes.
When designing a timetable for assessing the outcomes for a given major or, if needed, minor, degree program, consider the following questions:
How many outcomes has your department specified for that program? How often does your institution and/or accreditation
Accreditation A “process of self-review and peer review for improvement of academic quality and public accountability of institutions and programs. This quality review process occurs on a period basis, usually every three to ten years.” (Definition from CHEA .) The EP3 Initiative takes the position that physics programs should use the program-level assessment of student learning required for accreditation as an opportunity to engage in a cyclic process of program improvement and to design learning assessments that support goals for improving your program while satisfying accreditation requirements, avoiding duplicating efforts or doing busywork that doesn’t support your department. Nearly every college or university in the U.S. is accredited by one of the seven major accreditation agencies for four-year colleges and universities , and you need to know which one is your accreditor. Requirements vary among agencies, but all are concerned with learning assessment. The Council for Higher Education Accreditation (CHEA) is an association of degree-granting colleges and universities in the US that recognizes institutional and programmatic accrediting organizations. The Council of Regional Accrediting Commissions (C-RAC) is an association of accreditation agencies. Physics programs also need to ensure that physics courses for engineering and chemistry majors meet, respectively, the requirements of ABET , the accrediting body for most engineering programs in the U.S. (including some engineering physics and physics programs), and the American Chemical Society (ACS) , which approves chemistry degree programs in the US.
Do you wish to assess any outcomes two or more years in succession in order to help your department close the loop
Closing the Loop Using meaningful, effective assessment to improve a program through a continual, cyclic process embedded in the program’s practices and culture, rather than as an occasional activity that serves only to produce a required report. Such a process includes analyzing and reflecting on the results of assessments of program-level student learning outcomes, changing the curriculum to improve student learning, and subsequently re-assessing outcomes to determine the effect of your changes.
How many outcomes does your department have the capacity to assess each academic year? How can you ensure that your process does not overburden faculty with assessment responsibilities? Are there any outcomes that you always want to be assessed together in the same year, or conversely, any outcomes which you always want to assess in different years? (This may be desirable to help make assessment of certain outcomes more efficient by gathering similar information in similar courses, to compare the results of assessment of particular outcomes to one another, to distribute the assessment load more equitably among faculty, or simply to guard against stagnation.) Are there any outcomes or sets of outcomes that need to be assessed every year? For example, a department may define only a subset of the outcomes defined for the physics B.S. degree for a different program such as the physics minor. If each program must be assessed every year, at least one of the outcomes in this subset must be assessed each year. Is there any reason the cycle of assessment needs to be synchronized with the overall institutional requirement for how often outcomes must be assessed? Download all sample assessment timetables as a Word document
This timetable assumes a program that has six
program-level student learning outcomes
Program-Level Student Learning Outcomes Statements describing what your students should be able to do as a result of completing your degree program. Outcomes emphasize the integration and application of knowledge rather than coverage of material, and are observable, measurable, and demonstrable. They use specific, active verbs (e.g., “identify,” “develop,” “communicate,” “demonstrate”) rather than “understand.” Program-level student learning outcomes are often abbreviated as program-level SLOs or as PLOs , and are also known as program-level learning goals . The term “outcomes” is becoming preferred over “goals” or “objectives” because it makes it clearer that these are defined expectations upon completion of the program, rather than aspirational goals that may or may not be achieved. Examples include:
Identify, formulate, and solve broadly defined technical or scientific problems by applying knowledge of mathematics and science and/or technical topics to areas relevant to the discipline Develop and conduct experiments or test hypotheses, analyze and interpret data, and use scientific judgment to draw conclusions Communicate scientific ideas and results in written and oral form according to professional standards and norms Demonstrate and exemplify an understanding of ethical conduct in scientific and professional settings Program-level student learning outcomes generally focus on overall program outcomes, in contrast to course-level student learning outcomes, which are specific to the knowledge and skills addressed in individual courses. Accreditation requirements typically require program-level student learning outcomes to be defined separately for each degree program (e.g., BA, BS, or minor), even though there will often be considerable overlap among these sets of outcomes. For more details, see the section on How to Assess Student Learning at the Program Level . For examples, see the supplement on Sample Documents for Program-Level Assessment of Student Learning or the PhysPort expert recommendation How do I develop student learning outcomes for physics courses?
This timetable is designed so that
Two outcomes are assessed every academic year. Every outcome is assessed two years in a row. Certain outcomes are always assessed together. This timetable assumes a program that has six
program-level student learning outcomes
Program-Level Student Learning Outcomes Statements describing what your students should be able to do as a result of completing your degree program. Outcomes emphasize the integration and application of knowledge rather than coverage of material, and are observable, measurable, and demonstrable. They use specific, active verbs (e.g., “identify,” “develop,” “communicate,” “demonstrate”) rather than “understand.” Program-level student learning outcomes are often abbreviated as program-level SLOs or as PLOs , and are also known as program-level learning goals . The term “outcomes” is becoming preferred over “goals” or “objectives” because it makes it clearer that these are defined expectations upon completion of the program, rather than aspirational goals that may or may not be achieved. Examples include:
Identify, formulate, and solve broadly defined technical or scientific problems by applying knowledge of mathematics and science and/or technical topics to areas relevant to the discipline Develop and conduct experiments or test hypotheses, analyze and interpret data, and use scientific judgment to draw conclusions Communicate scientific ideas and results in written and oral form according to professional standards and norms Demonstrate and exemplify an understanding of ethical conduct in scientific and professional settings Program-level student learning outcomes generally focus on overall program outcomes, in contrast to course-level student learning outcomes, which are specific to the knowledge and skills addressed in individual courses. Accreditation requirements typically require program-level student learning outcomes to be defined separately for each degree program (e.g., BA, BS, or minor), even though there will often be considerable overlap among these sets of outcomes. For more details, see the section on How to Assess Student Learning at the Program Level . For examples, see the supplement on Sample Documents for Program-Level Assessment of Student Learning or the PhysPort expert recommendation How do I develop student learning outcomes for physics courses?
This timetable is designed so that
Two outcomes are assessed every academic year. Every outcome is assessed two years in a row. Outcomes that are assessed together alternate pairings. This timetable assumes a program that has three
program-level student learning outcomes
Program-Level Student Learning Outcomes Statements describing what your students should be able to do as a result of completing your degree program. Outcomes emphasize the integration and application of knowledge rather than coverage of material, and are observable, measurable, and demonstrable. They use specific, active verbs (e.g., “identify,” “develop,” “communicate,” “demonstrate”) rather than “understand.” Program-level student learning outcomes are often abbreviated as program-level SLOs or as PLOs , and are also known as program-level learning goals . The term “outcomes” is becoming preferred over “goals” or “objectives” because it makes it clearer that these are defined expectations upon completion of the program, rather than aspirational goals that may or may not be achieved. Examples include:
Identify, formulate, and solve broadly defined technical or scientific problems by applying knowledge of mathematics and science and/or technical topics to areas relevant to the discipline Develop and conduct experiments or test hypotheses, analyze and interpret data, and use scientific judgment to draw conclusions Communicate scientific ideas and results in written and oral form according to professional standards and norms Demonstrate and exemplify an understanding of ethical conduct in scientific and professional settings Program-level student learning outcomes generally focus on overall program outcomes, in contrast to course-level student learning outcomes, which are specific to the knowledge and skills addressed in individual courses. Accreditation requirements typically require program-level student learning outcomes to be defined separately for each degree program (e.g., BA, BS, or minor), even though there will often be considerable overlap among these sets of outcomes. For more details, see the section on How to Assess Student Learning at the Program Level . For examples, see the supplement on Sample Documents for Program-Level Assessment of Student Learning or the PhysPort expert recommendation How do I develop student learning outcomes for physics courses?
This timetable is designed so that
Two outcomes are assessed every academic year. Every outcome is assessed two years in a row. Outcomes are assessed together in all possible pairings. This timetable assumes a program that has four
program-level student learning outcomes
Program-Level Student Learning Outcomes Statements describing what your students should be able to do as a result of completing your degree program. Outcomes emphasize the integration and application of knowledge rather than coverage of material, and are observable, measurable, and demonstrable. They use specific, active verbs (e.g., “identify,” “develop,” “communicate,” “demonstrate”) rather than “understand.” Program-level student learning outcomes are often abbreviated as program-level SLOs or as PLOs , and are also known as program-level learning goals . The term “outcomes” is becoming preferred over “goals” or “objectives” because it makes it clearer that these are defined expectations upon completion of the program, rather than aspirational goals that may or may not be achieved. Examples include:
Identify, formulate, and solve broadly defined technical or scientific problems by applying knowledge of mathematics and science and/or technical topics to areas relevant to the discipline Develop and conduct experiments or test hypotheses, analyze and interpret data, and use scientific judgment to draw conclusions Communicate scientific ideas and results in written and oral form according to professional standards and norms Demonstrate and exemplify an understanding of ethical conduct in scientific and professional settings Program-level student learning outcomes generally focus on overall program outcomes, in contrast to course-level student learning outcomes, which are specific to the knowledge and skills addressed in individual courses. Accreditation requirements typically require program-level student learning outcomes to be defined separately for each degree program (e.g., BA, BS, or minor), even though there will often be considerable overlap among these sets of outcomes. For more details, see the section on How to Assess Student Learning at the Program Level . For examples, see the supplement on Sample Documents for Program-Level Assessment of Student Learning or the PhysPort expert recommendation How do I develop student learning outcomes for physics courses?
This timetable is designed so that
Two outcomes are assessed every academic year. Every outcome is assessed two years in a row. Either outcome 1 or outcome 2 is assessed every academic year. Outcomes 3 and 4 are assessed out of phase with outcomes 1 and 2, so that assessment of these outcomes alternates with simultaneous assessment of outcomes 1 and 2. This timetable assumes a program that has four
program-level student learning outcomes
Program-Level Student Learning Outcomes Statements describing what your students should be able to do as a result of completing your degree program. Outcomes emphasize the integration and application of knowledge rather than coverage of material, and are observable, measurable, and demonstrable. They use specific, active verbs (e.g., “identify,” “develop,” “communicate,” “demonstrate”) rather than “understand.” Program-level student learning outcomes are often abbreviated as program-level SLOs or as PLOs , and are also known as program-level learning goals . The term “outcomes” is becoming preferred over “goals ” or “objectives” because it makes it clearer that these are defined expectations upon completion of the program, rather than aspirational goals that may or may not be achieved. Examples include:
Identify, formulate, and solve broadly defined technical or scientific problems by applying knowledge of mathematics and science and/or technical topics to areas relevant to the discipline Develop and conduct experiments or test hypotheses, analyze and interpret data, and use scientific judgment to draw conclusions Communicate scientific ideas and results in written and oral form according to professional standards and norms Demonstrate and exemplify an understanding of ethical conduct in scientific and professional settings Program-level student learning outcomes generally focus on overall program outcomes, in contrast to course-level student learning outcomes, which are specific to the knowledge and skills addressed in individual courses. Accreditation requirements typically require program-level student learning outcomes to be defined separately for each degree program (e.g., BA, BS, or minor), even though there will often be considerable overlap among these sets of outcomes. For more details, see the section on How to Assess Student Learning at the Program Level . For examples, see the supplement on Sample Documents for Program-Level Assessment of Student Learning or the PhysPort expert recommendation How do I develop student learning outcomes for physics courses?
This timetable is designed so that
Two outcomes are assessed every academic year. All outcomes are assessed within a span of two years. 
