Looking for our new site?

Education

Trading Transparency and Accountability Today for Better Testing Tomorrow

September 23, 2013
Publication Image

2013-14: the school year all American students, in all public schools, were expected to be proficient in reading and math. It’s finally here. But don’t kid yourself – nobody expects American schools to meet that goal this spring. And thanks to the U.S. Department of Education’s ESEA waivers (and waivers of waivers), most won’t have to.

Instead, 41 states, Washington, D.C. and eight California school districts have different goals for student performance. Goals that cut achievement gaps, or delay the universal proficiency deadline, or lead to college and career readiness, or something else. In some states, failing to meet these goals – like failing to make AYP – triggers interventions as a priority or focus school. But in others, there isn’t any meaningful accountability attached to the new targets. Moreover, many states won’t name any new priority or focus schools this year.

While states don’t have a clean record when it comes to gaming accountability systems, that isn’t necessarily what’s happening here. Holding priority and focus lists constant from 2013 to 2014 is a pragmatic decision on states’ parts, because school performance goals aren’t the only thing changing. Across the country, in waiver and non-waiver states alike, students will also be field testing the new Common Core-aligned tests developed by SmarterBalanced and PARCC. And the Department is letting all states apply for additional flexibility so that some students, in some schools, would take the field test and wouldn’t take state standardized tests in at least one subject. In other words, states won’t have to “double test.”

This creates a problem for the continuity of school accountability systems. How can you judge school performance fairly and accurately when the data aren’t comparable between schools? That’s like determining the faster runner when one competitor has hurdles in their lane and the other doesn’t. Further, the field tests aren’t designed to be used for making school accountability determinations, or frankly, measuring student performance. As Tom Kane notes in his incredibly smart take on the issue, the field test is meant to test the validity of individual test items, not produce a valid score for an individual student. Making a trade-off between high-stakes consequences for school accountability and a valid field test seems like a relatively easy choice. Hold accountability determinations steady and continue all current school improvement efforts, but make sure the field test is conducted to the highest possible standard so that the tests are ready for primetime in 2015.

But it isn’t quite that simple. After all, accountability is about more than being labeled a “failing” school or a priority one. Accountability relies first and foremost on transparent, accurate reporting of student achievement data. And this is where the field test creates a much more harmful trade-off. The U.S. Department of Education will still require all students to be assessed in both reading and math, but states will not be required to publicly announce the results for students taking a field test. For the first time in the NCLB era, there will not be achievement data available for a significant number of students and public schools.

This is a big deal. These data (should) inform nearly every decision made in education – for families, for educators, and for policymakers. Should we send our child to the neighborhood school, or try to enroll her in a charter school nearby? How effective was our new 7th grade math curriculum? Did our new professional development program improve teaching quality? Are the interventions in our focus schools working? All of these questions will be much more difficult to answer without student assessment data. Further, as Bellwether’s Chad Aldeman and Andy Smarick write, this compromises efforts to measure student achievement and growth as required in Race to the Top, ESEA waivers, and a host of other Obama education reforms. Yes, states could continue to administer their current tests during the field test, as they have during previous assessment revisions. But many will not. While the Department was clearly trying to appease (or even subtly encourage) states to participate in the field test, would states have really balked at field testing if every student was also given the state assessment?

Giving up a year of meaningful school accountability is a high price for getting better, more rigorous assessments that reflect what truly matters: whether students are ready for college and career. But the Department didn’t just give up meaningful accountability. They’re also giving up public reporting of test results at the same time. Does that make the price too high?

While it’s too late to reverse the Department’s decision, states participating in the field test should take a prudent and limited approach to it. Let field tests be field tests. They weren’t designed to be used as measures of individual student achievement or school performance. And the more students and schools participating in field testing, the larger the effect on transparency and accountability will be. Unfortunately, a few states – most notably California – are already gearing up to scrap their state assessments entirely this year and only administer the field test.

2014 was never going to be the year we saw universal proficiency. Unfortunately, it could shape up to be the year we see universal missing data. Let’s hope other states don’t follow California’s lead.

What Might Ratings-Based Financial Aid Look Like?

September 18, 2013

Last month, President Obama stood before a crowd at the University at Buffalo to propose a new higher education affordability initiative. The plan calls for the U.S. Department of Education to rate colleges prior to the 2014-15 academic year. Then the Department would tie financial aid to those ratings by 2018 – a carrot-and-stick approach to college quality. But we wonder if the Department’s version will really have the teeth to penalize bad actors, and how feasible it really is.

So far, there’s not much information on the White House’s plan. For the most part, all we have to go off of is a White House fact sheet that summarizes the plan. According to the fact sheet,

“Over the next four years, the Department of Education will refine [the ratings], while colleges have an opportunity to improve their performance and ratings. The Administration will seek legislation using this new rating system to transform the way federal aid is awarded to colleges once the ratings are well developed. Students attending high-performing colleges could receive larger Pell Grants and more affordable student loans." [emphasis added]

There are a few items of note here. First, the White House acknowledges that any such effort will require congressional approval. That means that, at least without a sea change in the political environment, this may never come to fruition. But second, and more interestingly, the White House’s examples look at only the “carrot” side of the carrot-and-stick – more available aid for high-performing schools, without any clear punitive measures for poor-performing ones.

Of course, it’s far too early to say what implementation would look like. But it closely resembles an idea that the New America Foundation first published in Rebalancing Resources and Incentives in Federal Student Aid, and which Senior Policy Analyst Stephen Burd dug into deeper in Undermining Pell. Our proposal included both the “carrot” and the “stick” – a Pell bonus for high-performing schools that enroll a larger share of low-income students, and a Pell matching requirement for wealthy schools that divert aid away from low-income students.

The New America Pell Grant bonus differs somewhat from the administration’s. The administration plans to use a ratings system that will likely include a broad range of quality metrics; we would give the bonuses to public and private four-year schools that enrolled large shares of low-income students or to community colleges with strong student outcomes. Our Pell Grant bonus would be double the size of the maximum grant (currently $5,550).

We used data from the Federal Education Budget Project to calculate the costs and estimated the Pell bonuses alone (without the baseline costs of the Pell Grant program at these eligible schools) at $23.6 billion over 10 years for public and private four-year schools and $34.9 billion for community colleges. Those figures include schools that qualified for the proposed bonus based on 2010 data, as well as schools on the cusp of qualifying, which we assume would be willing to work a little harder for a substantial payoff.

At four-year schools, we found that the federal government already disburses $1.2 billion in Pell grant funding to already-qualifying schools,  and another $344 million to the 86 near-qualifiers. That made the math pretty easy – for the additional costs of the program over the baseline, we simply rounded up to provide a conservative estimate, and then counted up 10 years with built-in inflationary increases.

At community colleges, the math was a little trickier. We wanted to use quality metrics, a more simplistic version of those the Department of Education might use under a new rating system. It’s tough to see how community colleges are performing, though, because of limitations in the data. For example, the Department collects graduation rates only for first-time, full-time students, but public two-year colleges serve largely nontraditional students who don’t meet those qualifications. And students who transfer from a two-year to a four-year college without an associate’s degree are only marked as transfers, with no way to track them through the rest of their educational experiences.

Recognizing the data were so prohibitively absent as to keep us from finding a great measure, we calculated a combined graduation-and-transfer rate as a proxy. If the schools had a combined rate of at least 50 percent, they were eligible for a bonus. Many of the schools didn’t have good enough data for us to even arrive at a figure, but of the remaining schools, 262 were eligible, with Pell disbursements totaling $1.5 billion. We found another 120 who were close enough to qualify if they stretched a little further, and added their $1.2 billion in existing Pell money. We rounded up to $3.0 billion to account for missing and not-yet-successful institutions, baked in an inflationary increase, and added up the five- and 10-year costs. Again, those costs are in addition to, not including, the amount of Pell money that already goes to those schools.

Obviously, the New America proposal is not identical – or even similarly oriented – to the White House’s proposal. Ours focused on the needs of low-income students, not the quality of institutions (though with better data on colleges, a stronger focus on quality could be a rising tide that lifts all students).

But our proposal is instructive in a few ways. For one thing, the plan is going to be expensive. New America’s proposal, taken in total, is deficit-neutral, and we made up for the costs of the plan with savings from other proposals. Congress won’t be so lucky, and given the ongoing fiscal debates lawmakers are having, a plan that has one-year costs of upwards of $5 billion won’t be the most popular one. For another, the careful wording in the White House fact sheet means there’s no clear protection against bad behavior, at least in this part of the plan – just an incentive for good behavior. That may arguably be less effective than having both.

Any plan to tie financial aid to ratings is a long way off, and even the ratings system is a few years down the line. By 2018, we’ll have a different president, many different members of Congress, and undoubtedly new approaches to reforming higher education. It remains to be seen whether the plan will be strong enough to survive all that, or whether the 2018 political climate will actually be more amenable to these types of proposals. In the meantime, the New America Foundation will be watching for signs of life with this proposal, as well as the president’s other ideas.

Storify: Senate Passes Child Care Bill with Bipartisan Support

September 18, 2013
Publication Image

The Child Care and Development Block Grant (CCDBG) hasn't been reauthorized in 17 years. Today, the Senate Health, Education, Labor, and Pensions (HELP) Committee easily passed a bipartisan bill to renew the law, though it's still far from passage.

New Report Highlights Top Teachers’ Views on Education Policy

September 18, 2013

“Good teaching is hard to define, even for the profession’s most successful and reflective members.” So says a new report from The New Teacher Project (TNTP) that compiles highly effective teachers’ voices on issues ranging from their profession to hotbed education policy topics.

Early Educational Data Comments to U.S. Department of Ed

September 17, 2013

The Common Education Data Standards (CEDS) project is an initiative headed by the U.S. Department of Education’s National Center for Education Statistics to create comparable, consistent data definitions. It’s an entirely voluntary initiative, and its glossary includes preschool through workforce data elements. CEDS is in the process of refining its version 4 dictionary – open for comments through Friday, September 20.

New Report Shines Spotlight on Negative Effects of Pre-K Absenteeism

September 17, 2013
Allbriton PreK Classroom

This month is labeled the first-ever “Attendance Awareness Month” by the advocacy group Attendance Works, and there is plenty to which we ought to be paying attention. A 2008 study by the National Center for Children in Poverty (NCCP) estimated that one out of every 10 children nationally is chronically absent (meaning he misses at least 10 percent of scheduled days) in his first two years of school.

Aligning Investments in Parenting With Investments in Early Education

September 13, 2013

Alignment is critical in early education policy. That goes for curriculum, instruction, standards, and much more. To be highly effective, public early education programs need to be: 1) accessible to those who need them, 2) high-quality, and 3) aligned with the rest of the education system. The last part is certainly key; we know, for example, that pre-K programs work best when they are designed in tandem with the K–12 system into which they feed. However, it is a mistake to think of alignment as perfectly linear, running from pre-K straight through college admission. Students are also their parents’ children—and those parents’ influence can support or undermine educators’ work. Can targeted policies help align parenting with schooling? Should policymakers dare to try?

New Pell Grant, Federal Loan Data Reveal Changing Tides in Financial Aid

September 12, 2013
Publication Image

New data published by the U.S. Department of Education and by the Congressional Budget Office reveal changing tides in the American higher education system. And they uncover some interesting – and previously unknown – facts about federal financial aid.

The Department of Education’s Federal Student Aid (FSA) Data Center recently released data on the number of federal loan recipients and the total amount of loans disbursed in the 2012-13 academic year. (You can see those preliminary data by school in our easy-to-use database, the Federal Education Budget Project.) For the first time, the FSA data contain a breakdown between how much undergraduates and graduate students are borrowing, rather than rolling graduate and undergraduate Unsubsidized Stafford loans into one figure.

It’s a very important distinction. The data show that a whopping 41 percent of loan issuance in AY 2012-13 was for graduate and professional students. Meanwhile, graduate students were only 17.5 percent of all student loan recipients.

gradloanrecips.png   gradloandisburse2.png

Sources: New America Foundation, Federal Student Aid Data Center

Most people typically have undergraduates in mind when they think about the federal loan program, but in reality, the program is nearly as much about financing graduate studies as it is about undergraduate programs. Sure, graduate school can cost more than an undergraduate education, but that’s not necessarily why graduate loans feature prominently in the breakdown. Actually, it’s because the federal government does not limit how much graduate students can borrow for PLUS loans (and limits Graduate Stafford loans to $20,500 annually), but it imposes annual caps as low as $5,500 on undergraduates.

There’s another reason worth separating out loans for undergraduates and graduate students in the data. Under a bipartisan bill passed earlier this summer, interest rates will no longer be the same for Unsubsidized Stafford loans for graduates and undergraduates (both loan types had been set at 6.8 percent prior to AY 2013-14). Instead, starting this year, undergraduates will pay much lower interest rates on their loans (3.86 percent) than will graduate students (5.41 percent).

And that’s not the only news in federal student aid. Remember the Pell Grant funding cliff? For years we've heard about an impending drop-off in funding for the program – and it’s still baked into the budget, albeit a few years further out than first estimated. The Congressional Budget Office (CBO) reminds us of that looming funding cliff in a new report called The Federal Pell Grant Program: Recent Growth and Policy Options.

The CBO uses data on Pell grant aid and recipients to give policymakers an idea of what has driven costs in the past, what types of changes would reduce costs, and by how much. (Heads up: the biggest single cost-saver would be to allow only the lowest-income of the current Pell grant-eligible population to receive grants by requiring that they have a zero “expected family contribution” [$10.0 billion in 10-year savings], while the greatest cost would be increasing the maximum grant to $6,400 in AY 2014-15 [$5.3 billion over 10 years]).

Another interesting point in the CBO paper looks at the skyrocketing costs of the Pell Grant program. The big cost increases in the program in recent years owe a lot to community colleges. Much of the increase in the number of Pell recipients is due to a growing share of Pell students, more so than other factors, like growing enrollment. We wrote about that in 2011 after arriving at the same conclusion. Even so, Pell students still make up a far smaller share of total enrollment in community colleges than in the for-profit sector.

Both sets of data offer interesting insights into the growing and changing beast of federal student aid programs. The FSA data show the dramatically oversized influence of graduate and professional students in the distribution of loans, while the Pell data show the evolving nature of undergraduate aid. Both are work a close look as Congress returns to Capitol Hill and gets back to legislative business, so check out the Federal Education Budget Project to find your state or college.

Child Care Workforce Lacking in Opportunities

September 11, 2013

Federal data suggest that in 2010, the nation’s nearly 1.3 million child care workers earned an average of around $9.28 per hour, or $19,300 per year. The lowest-paid 10 percent of workers earned less than $7.65 per hour. With statistics like that, it’s no wonder studies have found that child care workers leave the profession at high rates -- according to one study, more than half of teachers who left the centers at which they worked actually left the occupation entirely.

Plausibility, Politics, and Process: What We Can Learn from Indiana and Tony Bennett

September 11, 2013

Definition of Plausible

  1. superficially fair, reasonable, or valuable but often specious <a plausible pretext>
  2. superficially pleasing or persuasive <a swindler… , then a quack, then a smooth, plausible gentleman — R. W. Emerson>
  3. appearing worthy of belief  <the argument was both powerful and plausible>

Last week, a report commissioned by the Indiana state legislature provided more detail on just exactly what happened a year ago as then-Superintendent of Education Tony Bennett prepared to release the state’s first A-F school grades – one of many high-profile reforms Bennett and his staff championed. The emails uncovered a last-minute scramble to change grades for certain schools and, ultimately, led to Bennett’s resignation as schools’ chief in Florida. But many unanswered questions remained. Did last week’s report answer them?

Here’s what we knew before:

  • Emails obtained by the Associated Press showed that the final grade for Christel House Academy and a dozen other schools with nontraditional configurations (e.g. grades 5-9, K-10) were changed internally prior to the public release of the grades. The emails showed Bennett and his staff were particularly concerned about an initial ‘C’ grade for Christel House, founded by a prominent political donor and regarded as a top-performing charter school. Christel House had recently expanded to offer high school, in addition to elementary and middle grades.
  • After reading the emails and analyzing the data, I reported that the grading change was only possible once performance data from these schools’ high school grades were eliminated from the formula. And I had a problem with that. The changes were made thanks to a “loophole,” in secret and without a public explanation. This made it near impossible for parents and families to know that Christel House’s K-8 grades earned the ‘A’ grade, but its high school did not.
  • Further reporting found that another 165 schools, including Christel House, benefitted from a second tweak to the formula: removing a cap on the number of bonus points schools could receive for high rates of student growth. This allowed elementary and middle schools with high growth in one subject to compensate for low performance in another subject area.

And here’s what the new report says: “In the end, Authors found that the two adjustments administered to determine Christel House Academy’s final grade were plausible and the treatment afforded to the school was consistently applied to other schools with similar circumstances.”

This isn’t news. That’s because the question wasn’t whether the changes were applied consistently – dozens of schools benefitted from the two changes, and there was never any indication that this wasn’t the case. The question wasn’t even if the changes were plausible. Of course officials could make some reasonable-sounding explanation for increasing the emphasis on student growth in the grading formula, or for removing the high school data for some schools… just as officials could also make a reasonable-sounding explanation for limiting the emphasis on growth, or for using all available high school data for all schools. In fact, that’s exactly what officials did with schools that only serve grades 9 and 10 (they just didn’t apply the same plausible logic to schools serving grades 5-10).

In other words, just because Bennett’s decisions were plausible – “superficially fair and reasonable” – it doesn’t mean they were right or in the best interests of students and families. The decision to ignore these schools’ high school data in the A-F system is like telling parents their child is an honor roll student, but only after tossing out a failing grade in Spanish because it’s their first year taking the language. Sure, it’s a plausible argument, but is it the right call? That's debatable, and it's a debate that should have happened in public.

The larger takeaway here isn’t just about the plausibility of these changes. It's also the process by which they were made. Would we be having this conversation if Bennett and his staff had been open about altering the formula and removing the high school data or the growth caps from the beginning? Or at least once the emails were released?

School accountability systems cannot function without public trust. Anyone – from parents, to policy analysts, to reporters – should be able to determine how a school’s grade was calculated. Students can determine why they earned a B+ on a math test by looking at which questions they missed and how many points they lost for each, just as the public should be able to look at a school’s ‘B’ grade, understand how it was calculated, and note the school’s strengths and weaknesses. And if changes are made to the grading rubric or the weighting of components, they must be announced and explained publicly – not buried in Excel files or internal emails.

Predictably, friends of Bennett have been quick to forgive, just as his political foes were once quick to judge. But these black-and-white pronouncements overlook many of the valuable lessons that can be learned from the report – as the authors note, their work neither condemns, nor vindicates Bennett.

The report does confirm how the grading changes occurred. Moreover, it lays out several useful recommendations for Indiana’s A-F school accountability system, notably increasing the transparency of the decisionmaking process, improving capacity within the state education agency to handle the technical aspects of A-F development, and piloting school grades before full implementation. And as PoliticsK-12 reported, these lessons extend beyond Indiana to every state updating its accountability system under ESEA flexibility. Instead of judgment and vindication, let’s also talk about how these accountability systems can be improved. It’s time to focus on the process, not just the politics.

Syndicate content