Effective Practices
Engage your department and institution in developing a plan for the review process
- Learn about the institutional processes and templates for program reviews, the rationale and logistics of the , and the role the review plays in guiding your department.
- Ensure the aligns with institutional goals and procedures and includes engaging your department, gathering and analyzing data, connecting to other reviews and assessments, reflecting on advice, setting goals, and planning initiatives.
- Understand the timeline for the review and its associated logistical steps, e.g., identifying and recruiting , formatting and submitting a self-study report. See the resource on Program Review Timeline for an example.
- Identify how results of the will be used by your administration in allocating resources and by your department in improving its program and student success.
- Identify and discuss the benefits of completing a . For example, reviews can help your department celebrate and promote its accomplishments and identify areas for improvement. Frame the review as an opportunity to improve your department rather than as an onerous requirement.
- Establish roles and expectations for how each member of your department will contribute to the .
- Read documents associated with the previous review and gauge progress toward attaining goals, objectives identified as a result of that review, and outcomes of other initiatives undertaken since then.
- Challenge the group to identify department strengths and weaknesses (e.g., through a or ) and a preliminary list of issues and topics to explore with regard to your department, its programs, and its students. See the resource on Topics a Department Might Explore in a Program Review for more details.
- Align your review with institutional program review processes, external review processes (e.g., ), data gathering processes associated with institutional accreditation, institutional budgeting and planning conversations, assessment of student learning, and other data gathering processes.
- Educate department members about the values of a robust program review process. If concerns about engaging in a program review arise, see the section on How to Be an Effective Chair for guidance on how to address challenging situations.
- Identify and recruit a group that represents the diverse perspectives and concerns of your department, e.g., recruiting and retention; equity, diversity, and inclusion; curriculum; student progress; assessment; program strengths and how to sustain them; research programs and directions; faculty, student, and staff concerns.
- Consider including staff or students in the group.
- Recognize the potential for unequal power dynamics (between, e.g., faculty members of different ranks, individuals from and others, faculty and staff or students) or conflicts of interest in the group.
- Agree on group leadership.
- Task the group with exploring topics, addressing questions, raising issues of concern with your department for discussion, and regularly updating department colleagues through appropriate means, e.g., departmental meetings and newsletters.
- Construct a concise plan for how the will be conducted and used to improve your department and contribute to the broader mission of your institution.
- If necessary, advocate for resources needed to conduct a (e.g., stipends, travel and meal costs, staff support), identifying how this investment can assist your department in improving its programs and course offerings, through, e.g., insights and potential strategies derived from conducting the self-study and from recommendations made by .
- Consider requesting additional support for members of your department facilitating the , e.g., fulfillment of service-load requirements, resources to address identified departmental needs, professional development funding.
- Meet with upper-level administrators (e.g., your dean and/or provost) to present the aforementioned plan, possible outcomes, and budget; and to request input on how to define the charge for and selection of the reviewers, e.g., processes for selecting and engaging , perspectives on issues to explore and address, and insights on how the results of the will be used by your institution.
- Facilitate development of a timeline (see the resource on Program Review Timeline for an example), complying with any established institutional deadlines.
- Collaborate with upper-level administrators to determine a window of available dates for the visit by .
- Set a completion date for the self-study report, at least a month before the ’ visit, that allows for unexpected delays while ensuring reviewers have at least two weeks to review the report.
- Establish a timeline for completing important milestones in the , e.g., collection of new data to address emerging questions if necessary, merging of data from previous annual reports, analysis and reflection, drafting of the self-study report, departmental discussions of the draft.
- Schedule departmental faculty and staff discussions for initial planning, subsequent reviews of data, review of the draft self-study report, and development of an action or strategic plan.
- Familiarize your department with institutional requirements and procedures for identifying and selecting .
- Identify through departmental discussions the perspectives, experiences, and demographics that an external review team needs to enable a comprehensive review of critical issues and topics of interest for your department. For example, consider current and former chairs of peer or aspirant departments, reviewers used by peer departments, those with training as external reviewers, those who have faced similar challenges, and suggestions from staff at professional organizations such as or .
- Look beyond your existing networks to broaden the pool of potential reviewers to recruit a more diverse group. See the guidance on how to build a strong and diverse candidate pool for faculty and other instructional staff positions in the section on How to Be an Effective Chair for suggestions that can also be applied to recruiting reviewers.
- Consider the value of including a member from a previous external review team in the team of reviewers to provide perspective on how your department has acted on previous recommendations.
- Avoid having collaborators and coauthors of department members serve as reviewers, as their relationship to your department members may reduce the credibility of their report.
- Ensure are engaged as early as possible (ideally at least three months) before their visit, with their commitment confirmed in writing. (See the resource on Sample MOU for Engaging an External Reviewer for Physics Program Review for more details.)
- Hold a video conference with the external review team as early as possible (ideally at least three months) before their scheduled visit to discuss the , how recommendations will be acted upon, your institution’s and department’s goals and objectives with respect to the review, the site visit agenda, the content of the self-study report, and issues and questions your department would like to address.
- Confirm the visit dates and desired outcomes with key members of your administration, department, and .
Develop a self-study report
- Discuss charges to the internal review committee from your department and administration.
- Study your department’s previous self-study report, ’ report, most recent departmental and institutional strategic plans, current initiatives, and ancillary materials, e.g., recommendations from visiting committees, task forces, department retreats, and administrators.
- Refine your department’s list of issues and topics to explore, identified during initial department discussions (see 1.b), e.g., increasing enrollment, improving departmental culture and climate, improving equity and diversity, improving student learning outcomes, strengthening student and faculty recruiting and retention, serving institutional accreditation goals, supporting the general education program, obtaining increased external funding, educating and graduating increased numbers of science teachers to meet local needs, raising the visibility and reputation of your institution and department. See the resource on Topics a Department Might Explore in a Program Review for more details.
- Review the timeline required to complete the self-study report.
- Delegate assignments (e.g., gathering and analysis of data, investigation of peer and aspirant departments, writing of self-study report sections, arranging of the site visit), and ensure the workload is shared equitably among group members.
- Identify processes and key stages and dates for keeping other department members (faculty, staff, and students) informed and involved.
- Draft an organizational scheme for the self-study report (if your institution does not provide or require one). See the resource on Template for a Self-Study Report as an example.
- Identify, collect, and analyze data and materials that will be needed to address the issues and topics you identified while preparing the self-study report, collaborating with your office of institutional research as appropriate and/or required by your institution.
- Consider soliciting anonymous feedback from department members (faculty, staff, and students) to better understand internal department dynamics.
- Identify and collect supporting data and documents that provide context and present a comprehensive picture of your department, its students and faculty and their activities (e.g., enrollment patterns, student persistence in the major, summary of programmatic and curricular assessment, course syllabi, teaching assignments, faculty and staff annual reports, two-page faculty CVs), and its resources (e.g., facilities, equipment, budget). See data sections of the resource on Template for a Self-Study Report for more details.
- Reach agreement on how to analyze data and assign individuals to gather and analyze subsets of the information by a specific deadline.
- Establish a process to gather and analyze these data on an ongoing (e.g., annual) basis to facilitate departmental conversations and to lessen the workload associated with periodic review.
- Identify peer and aspirant departments and review this list with your department and administration (e.g., your dean or office of institutional research) to validate choices and align them with your institution’s own list of peer and aspirant institutions.
- Gather data from outside sources (e.g., the AIP Statistical Research Center or the web page How Does Your Institution Compare?) to provide context and comparison for your department.
- Identify key characteristics for comparisons, e.g., numbers of majors, graduates, faculty, and staff; curricular offerings; equipment and supplies budget; strengths; challenges.
- Collect and present data covering the entire period since the previous external review in order to evaluate trends.
- Reach agreement on how to analyze data, determine who will gather and analyze information, and set a specific deadline.
- Remind the committee that the purposes of preparing the self-study report are to (1) reflect on the past, (2) compare your department to peer and aspirant programs, and (3) think strategically about the future of your department, including identifying potential challenges, opportunities, and objectives. This process can be messy, with competing ideas and considerations of institutional, departmental, and personal priorities and constraints.
- Confirm the organizational scheme for the report and assign committee members to draft specific sections by agreed-upon deadlines. For a description of the suggested content, see the resource on Template for a Self-Study Report.
- Discuss analyses performed; prior initiatives and their outcomes, strengths, challenges, and questions; and potential new initiatives, in light of your department’s mission, vision, and previous action or strategic plan.
- Create a coherent synthesis of the draft sections, putting the most important issues and information at the beginning and using graphical representations of data where appropriate.
- Identify possible future objectives for your department and potential actions to achieve them, being as specific as possible to tap the expertise of external reviewers and members of your administration.
- Share and discuss the draft with faculty, staff, students, and other stakeholders, emphasizing key points, e.g., proposed changes to your department’s goals, objectives, and draft actions; critical challenges and initiatives to address them; issues and topics for to explore; topics to discuss with your administration.
- Revise the report based on feedback from departmental discussions.
- Proofread the final document for errors, word choice, and grammatical issues.
- Distribute the report to department members, administrators, and reviewers, as aligned with the timeline. See the resource on Program Review Timeline for more details.
Plan and host a site visit
- See the resource on Sample Agenda for an External Reviewers’ Site Visit.
- Schedule time for the reviewers to confer privately at the start of their visit, e.g., during the first evening, after arriving in the afternoon.
- Plan an opening conversation between the reviewers and the leader of the internal review committee to identify key issues your department would like addressed (e.g., budget; staffing; enrollments; concerns raised by faculty, staff, alumni, and administrators) and to answer ’ questions.
- Identify times early in the visit when relevant upper-level administrators are available to provide administrative perspectives and expectations for the review, and times at the end of the visit for a debriefing from reviewers.
- Identify times when department members are available for individual and group meetings and plan for as many meetings with various individuals and groups (e.g., separate meetings with students, pre-tenure faculty, tenured faculty, non-tenure-track , other staff, members of , and representatives of departments that require physics service courses) as the schedule permits.
- Include time for reviewers to share separate experiences with each other, reflect during the visit on their conversations, transition between meetings, take breaks, check messages, and prepare their oral and written reports. (See the section on How to Serve as an Undergraduate Program Reviewer for details about oral and written reports.) Accommodate any special needs, e.g., extra breaks for pumping or other needs.
- Ensure there is a final exit meeting with your department followed by a separate meeting with your administration to report out key findings and recommendations.
- Prepare students, faculty, and staff in your department for the reviewers’ visit, e.g., discuss the scope and purpose of the review, its connection to your institution’s processes, and use in subsequent strategic planning; encourage open and honest discussions with reviewers.
- Be prepared to address disruptions of reviewers’ travel plans or last-minute cancellation of a reviewer’s participation.
- Consider offering refreshments or other incentives to ensure good attendance at student meetings and ensure their timing does not conflict with common class times and accommodates both resident and commuter students.
- Confirm reviewers’ travel arrangements to ensure appropriate arrival and departure times to and from campus.
- Plan local transportation, lodging, and meals, taking into account special needs, e.g., accessibility and dietary restrictions.
- Identify steps in payment arrangements and plan for those in advance.
- Make arrangements for on-site meetings, reserving a private, lockable space for the external review team to meet and spaces for the team to meet with students without faculty or staff present.
- Circulate the final visit agenda to all department members and everyone participating in the visit.
- Arrange for how reviewers will find their way between meeting locations.
- Arrange internet access and other work materials (e.g., notepads) for reviewers.
- Check in with reviewers as their visit progresses, and accommodate, if possible, requests for schedule adjustments or additional information.
- Reconfirm arrangements for reviewers’ departures and their plan for submitting the final report, and ensure all paperwork is in place for paying stipends and travel costs.
- Provide feedback to reviewers if elaborations in their written report are desired on points mentioned in their oral report, if their oral report failed to address critical topics on which your department wanted advice, or if the oral report included factual errors.
Enact subsequent planning actions
- Promptly hold a meeting with faculty and staff following receipt of the ’ written report to discuss recommendations and note feedback from various groups.
- Consider inviting students or other stakeholders to discuss recommendations and possible initiatives.
- Use these gatherings to think positively about how the report can be used to develop an action or strategic plan to improve your department.
- Understand how the report progresses through the various levels of internal institutional review.
- Draft a departmental written response to recommendations made by and circulate this document as broadly as possible, including to all faculty and staff in your department and key administrators.
- Respond promptly to inquiries from any internal groups evaluating the self-study, external review report, and departmental reply in a responsive way.
- Prepare thoroughly for any meetings with upper-level administrators to discuss outcomes, e.g., review all documents, meet in advance with department members to discuss messages to convey, and ensure all department members are prepared to advocate coherently for department needs.
- Establish a date for completion of a new or updated action or strategic plan that includes specific initiatives in response to the review process, as agreed upon by your department and upper-level administrators.
- Make a plan with your department and your administration to regularly (e.g., annually) review your department’s progress in light of the report.
- For a description on how to create (or revise) and use a department action or strategic plan, including how to publicize the document and results achieved, see the section on How to Create and Use a Strategic Plan, starting with Effective Practices 1.D.
- Plan for the next review by establishing processes for gathering necessary data on an ongoing (e.g., annual) basis and aligning these processes as far as possible with other review processes such as assessment of student learning, institutional budgeting and planning processes, institutional accreditation, and external accreditation (e.g., ) requirements.