Planning for Proficiency: Verification of Performance Indicators

This is the fifth brief in a thirteen-part series designed to inform Maine school leaders as they work to develop and implement their proficiency-based learning system.

Download this brief as a PDF
Watch a short video outlining these resources

What you need to know

  • A proficiency-based system uses criterion-referenced assessments.
  • There are a variety of performance indicator verification approaches schools can use.
  • In a proficiency-based learning system students are provided with multiple opportunities to complete assessments.
  • Adopting or adapting high quality work from other districts may be very helpful.

What you need to know

  • Develop, adopt, or adapt 8–10 performance indicators for each graduation standard.
  • Develop, adopt, or adapt high quality scoring criteria.
  • Provide time for teachers to create, revise, and calibrate their scoring of common assessments.
  • Lead the district in determining which method teachers will use to verify student achievement of performance indicators.
  • Select or update electronic gradebooks to reflect the verification system.

Timeline
Spring 2017

Reminder: The steps we recommend and the resources we provide are grounded in the PBL Simplified Model we have created and assume a general level of familiarity with it.

Performance indicators support a criterion-referenced assessment that measures student learning against a set of agreed upon learning standards. In a proficiency-based learning system, a wide variety of criterion-referenced assessments are used to individually assess students against clear, universally-shared performance indicators. Students are not compared to each other, but rather to how they, as individuals, measure up to the learning expectations. Keep in mind that schools may currently assess students by comparing them to their peers (norm referenced), and take into account various behaviors, attendance, homework completion, and class participation. In a proficiency system, students are not disadvantaged by prior poor performance on assessments as they are given multiple opportunities to demonstrate proficiency.

Using Scoring Criteria

Performance indicators outline what students need to know or be able to do; scoring criteria outline how well students must perform to achieve proficiency. Using common scoring criteria to assess performance indicators requires a new mindset for all of us. Professional development to ensure consistent application of scoring criteria, including calibration of scoring student work, will ensure the assessment system is fair and accurate from the outset. Schools may adopt or adapt scoring criteria from other districts that have invested considerable time in this work.

Given high-quality graduation standards, performance indicators, and scoring criteria, the real work begins when teachers gather to review student work that reflects each level of scoring criteria. Proficiency on any performance indicator should eventually have common meaning throughout the school. Because building common agreement is so important, it is wise to use collaboration time to develop a shared understanding of proficiency and the scoring process. As with scoring criteria, some districts have chosen to adopt or adapt work from other districts rather than writing all new performance indicators and scoring criteria.

For those schools able to allocate time in writing scoring criteria, the Great Schools Partnership offers advice on best practices (see Scoring Criteria Design Guide). These task-neutral (applicable across diverse assessments) criteria should be increasingly complex in cognitive demand, focus on the quality of work produced, and describe what students can do. Further, they should be applicable to a variety of learning experiences, not one particular project or task, keeping in mind that, over time, results from performance indicator-aligned assessments are aggregated to determine proficiency levels on graduation standards. The scoring criteria design process takes practice and collaboration among teachers, with attention to design principles that help build shared agreement and consistency. Classroom assessments may apply to several performance indicators and standards, so the scoring of these indicators across courses must be consistently applied. This consistency ensures that the scores on performance indicators are accurate and fair measures of student achievement.

Verification of Performance Indicators

There are actually two verification steps used in a proficiency-based learning system. First, throughout a grading period, teachers accumulate summative assessment scores of student achievement on performance indicators. The collection of these values is aggregated to generate a score that represents student achievement on each performance indicator. Second, the graduation standard score is then derived using the collection of aggregated scores from the performance indicators related to that standard. This information brief focuses on the first step in this process, the aggregation of individual performance indicators.

There are three options that may be used to determine an aggregated performance indicator score. These are described in detail on the GSP website: decaying average, power law, and most recent score. (Note that power law and decaying average may require specialized online gradebook systems.)

Decaying average formulas assign progressively decreasing weight to older scores, thus newer scores are more heavily weighted in the final score. Since student learning improves over time—and some students may experience early struggles—this method could encourage teaching practices that focus on learning growth and be less discouraging for the student. Additionally, since earlier scores do actually count, if only minimally, it provides an incentive for students to work hard throughout the learning process. This method works best when there are many scores to consider within any grading period.

Power law (also known as trending) plots assessment scores over time and generates a score that is a “best fit.” It answers the question: What score would the student most likely receive on the performance indicator if she were assessed again? An advantage is that this method does not penalize students for early poor performance and produces scores that reflect what the students knows at the end of the term. However, because of the way the formula itself is written, it’s possible to produce a final score that is higher than the highest one earned by the student. For this reason, the Great Schools Partnership recommends decaying average as a better method.

Using the most recent score or scores to determine if students have achieved performance indicators more accurately reflects knowledge acquired over the course of the grading period, and will encourage students to improve their performance. Students who meet standards early on need to maintain high performance throughout the term. While no system is perfect, using the most recent score allows students the opportunity to recover from past failures and not be disadvantaged by earlier struggles. It is also more equitable for struggling learners. Further, students who meet standards early on need to maintain high performance throughout the term. Calculating final scores in a way that assigns the greatest weight to the most recently assessed work is ultimately a truer measure of what a student actually knows and is able to do.

Alternative methods used by some schools include mean, mode, and highest score. Though most familiar to teachers, averaging scores on the same performance indicators may defeat the purpose of a proficiency-based system. Progress over time is unclear as is the level of achievement at the end of a term when teachers have to make a determination regarding moving forward in a course. Using the mode or most common score may not accurately measure the depth of learning as the struggles along the way would factor more heavily in the final grade. Alternatively, counting only the highest score would not take into account a student whose effort falls off toward the end of the term, providing an incomplete picture of work habits.

The Great Schools Partnership has developed resources that offers guidance and thoughtful explanations of these various methods of verifying proficiency. It is essential for teachers to first have a shared understanding of what proficiency means for each performance indicator before determining which method to use, and to also understand the limitations of each method. The choice of verification approaches will impact many elements of teaching and learning. Schools will need to make a range of decisions, including: consequences for missing or late work; how, and whether, to record progress on formative assessments; and the expected pace of progress, to name a few. Resolving these important issues will be crucial for both teachers and students when the verification of proficiency system is in place. Ultimately, we want parents to have confidence that scores from summative assessments are true measures of what a student knows and is able to do and, in particular, ensures college- and career-readiness for all.

Resources

 

Creative Commons License Proficiency-Based Learning Simplified by Great Schools Partnership is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.