An Ocean of Unknowns

Risks and Opportunities in Using Student Achievement Data to Evaluate PreK-3rd Grade Teachers
May 15, 2013 |
Publication Image

Click here to view PDF

What is the best way to use data to measure teacher impact on student learning? States and school districts are attempting to navigate these uncharted waters. As of 2012, 20 states and DC require evidence of student learning to play a role in evaluating teacher performance. As a result, better information on student learning is in high demand, and no grade level is immune. Historically, most states have required standardized testing only in grades three through eight. But now those 21 states, with likely more to follow, must figure out comparable ways to measure student learning in the “untested grades,” as well, including pre-K, kindergarten, and grades one and two. And even with testing in grade three, a lack of baseline data has implications for those teachers too.

Determining growth measures for these grades is among the most complex pieces of teacher evaluation reform. In this early stage of life, children’s developmental growth—their acquisition of physical, cognitive, and social-emotional skills; their base of general knowledge; their strength of persistence and motivation; and their language and literacy ability—is directly linked to their academic growth. So measures of student learning should account for how young children actually learn and measure more than just reading and mathematics if we are to obtain an accurate picture of a teacher’s impact on her young students’ learning.

This paper provides a snapshot of how student achievement data are being used in teacher evaluation systems today and illuminates the issues causing states and school districts the most struggles. Most states are using one of or some combination of three approaches: student learning objectives, shared assessments, and shared attribution. The Early Education Initiative at the New America Foundation examines these approaches in five states (Colorado, Delaware, Florida, Rhode Island, and Tennessee) and three school districts (Austin, Texas; Hillsborough County, Florida; and Washington, DC). Each of the approaches carries its own risks and opportunities.

Reasons for Reforms of Teacher Evaluation Systems

Sources: New America Foundation; National Council on Teacher Quality; Education Week; U.S. Department of Education; and Center on Great Teachers and Leaders at the American Institute for Research. (Based on information as of 2012.) This map was updated on July 29, 2013 to reflect new information.

The first approach, student learning objectives (SLOs), centers on a teacher's students. The teacher—with his or her administrator—creates a measurable objective, identifies an assessment to measure that objective, and establishes a challenging but attainable target for students.

Opportunities with SLOs:

  1. They foster school-level collaboration and shared priorities.
  2. They can help improve instruction.
  3. They can help teachers better meet individual student needs.
  4. They can support a more well-rounded curriculum.
  5. They attain teacher support.

Risks with SLOs:

  1. They are resource-intensive to develop.
  2. There is limited expertise at the district- and school-levels.
  3. They come with an inability to compare teachers.
  4. They come with a high potential for manipulation.

The second approach is creating or identifying shared assessments at the district or state level.

Opportunities with Shared Assessments:

  1. They facilitate comparisons across schools and districts.
  2. They could build skills transferrable to the classroom.

Risks with Shared Assessments:

  1. They require significant financial and time resources to develop.
  2. There are too few appropriate assessments.
  3. They could lead to curriculum narrowing and teaching to the test.
  4. There are important concerns about test security.

The third approach, shared attribution, uses a school-wide, value-added score. Typically this is based on results from evaluations from third to fifth grade, such as third grade reading scores on a state’s standardized test to determine the growth rating for a kindergarten, first, or second grade teacher.

Opportunities with Shared Attribution:

  1. It promotes shared accountability.
  2. It uses existing resources.

Risks with Shared Attribution:

  1. It does not help to provide useful individualized information to teachers.
  2. It does not help to differentiate teachers in a meaningful way.
  3. It does not measure a teacher’s impact on her own students’ learning.

To maximize the opportunities and minimize the risks, the New America Foundation’s Early Education Initiative makes three recommendations and poses nine considerations for state and district policymakers as they move forward with this work.

Recommendations

  1. Account for specific attributes of PreK-3rd teachers.
  2. Pilot and evaluate.
  3. Do not use “shared attribution” measures from later grades as the sole measure of student growth to evaluate early grade teachers.

Considerations

  1. Assessments are designed to be used in specific ways and do not always lend themselves well to other purposes.
  2. States will need to decide whether there should be one statewide system or many different district-level systems, and be prepared to provide technical assistance to discover what measures are appropriate for young children, what skills should be measured, and how to measure them in accordance with developmentally appropriate guidelines.
  3. While state and district officials may focus on improving numeracy and literacy in PreK-3rd, they should be concerned with whether students are developing crucial skills in the other domains of learning.
  4. Engaging schools of education in conversations about teacher evaluation is important, so prospective teachers and principals can gain expertise in assessment models.
  5. Creating a system in which teachers set goals and design measures to gauge student growth when their compensation or jobs depend on the results is rife with problems, as with SLOs.
  6. Different delivery models of pre-K and kindergarten make it difficult to tie student growth to individual teachers in the earliest grades.
  7. States should align their teacher evaluation systems with the Common Core State Standards before implementing the new assessments in the 2014-15 school year.
  8. In evaluations of PreK-3rd grade teachers, states and districts should consider whether teachers administered student assessments appropriately and what they did with the data.
  9. Since there is limited research on the approaches discussed in this paper, states and school districts should proceed cautiously in selecting assessments for measuring student learning in the early grades.

Regardless of the challenges states face in overhauling teacher evaluation systems, getting it right is crucial in the PreK-3rd grades. Research has confirmed, time and time again, that the quality of instruction and the quality of learning opportunities in children’s formative years sets the foundation for their success as students, and, later, their success as adults.

Read the full report here.

Related Files