How to Hold Colleges and Universities Accountable for Prices and Outcomes

Key Findings

• The federal government funds tens of thousands of higher education programs that yield zero or negative return on investment (ROI). The most important factor is field of study.
• Borrowers who graduated from programs with low or negative ROI are less likely to pay back their federal student loans.
• Existing outcomes standards for federally-dependent programs, such as the Cohort Default Rate, fail to hold most negative-ROI programs accountable.
• In this paper, we propose seven principles for the design of stronger outcomes-based accountability policies.

Introduction

Students almost always say their primary reasons for going to college are expanding their employment opportunities and increasing their earnings potential. If higher education always succeeded in significantly improving students’ earnings power, almost no one would have trouble paying back their student loans. But over 7 million federal student loan borrowers are in default, and many millions more are either delinquent or not making progress paying down principal.

Through federal grant and loan programs, taxpayers fund over 200,000 unique degrees and certificates and colleges across America. But despite the federal government’s stamp of approval, not all of these programs provide their students with a positive return on investment (ROI). Many programs do not increase their students’ earnings enough to justify the costs of college–and the debt that students take on to finance them.

If higher education always succeeded in significantly improving students’ earnings power, almost no one would have trouble paying back their student loans. But over 7 million federal student loan borrowers are in default, and many millions more are either delinquent or not making progress paying down principal.

The lack of financial return associated with tens of thousands of degree and certificate programs is a major contributing factor to low student loan repayment rates. Borrowers with low incomes are far more likely to default on their loans. When the government funds negative-ROI programs, students suffer with loans they can’t afford and degrees that weren’t worth the cost, while taxpayers must cover the losses on loans that students do not repay.

More accountability for higher education programs dependent on federal funding is sorely needed. This report makes the case that accountability should aim to ensure that students receive a positive return on investment from their education and proposes seven principles that should form the foundation for accountability policies at the federal level.

A review of return on investment (ROI)

FREOPP has published estimates of ROI for tens of thousands of higher education programs dependent on federal funding, including bachelor’s degrees, graduate degrees, and sub-baccalaureate credentials. Our definition of ROI is the increase in earnings that the median student can expect from her education, minus the direct costs (tuition and fees) and indirect costs (lost earnings while out of the labor force). In our preferred measure, we adjust our estimates of ROI to account for the risk that students will not complete their programs and thus fail to realize the associated earnings gains.

We estimate that 28% of bachelor’s degrees have negative ROI, meaning the degree does not increase students’ earnings enough to justify the costs of college and the risk of dropping out. The share of nonperforming programs is higher for undergraduate certificates, associate degrees, and master’s degrees. Professional degrees such as law, medicine, and dentistry tend to provide positive ROI.

We estimate that 28% of bachelor’s degrees have negative ROI, meaning the degree does not increase students’ earnings enough to justify the costs of college and the risk of dropping out.

The most important factor correlated with ROI is field of study. Labor market demand for graduates trained in certain fields is much stronger, and that is reflected in graduates’ earnings. Large differences in ROI can exist even within the same school. For instance, the University of Pennsylvania’s bachelor’s degree in finance boosts its students’ net lifetime earnings by more than $4 million. But the school’s bachelor’s degree in film has negative ROI.

Gaps in ROI between different fields of study also exist at other credential levels. Postsecondary certificates in vehicle maintenance or licensed practical nursing are far more likely to pay off than those in cosmetology. A master’s degree in computer science typically has positive financial value, while most MBAs and MFAs have negative ROI. Often, a degree or certificate from a non-elite school in the right field has higher ROI than a degree from a prestigious school in the wrong field.

All the programs included in the ROI calculations have one thing in common: they are funded by the federal government. In the 2019–20 academic year, the Department of Education disbursed over $90 billion in loans to students attending hundreds of thousands of higher education programs across the country. Additionally, taxpayers supplied nearly $30 billion in Pell Grants, $13 billion in education tax credits, $13 billion in state-based grants, and $1 billion in federal work-study. Not all of those dollars have funded equally valuable programs.

Why higher education needs accountability

Non-repayment is one of the most critical issues facing the federal student loan program. In the last quarter before the Covid-19 payment pause began, 7.7 million borrowers with federally-held student loans were in default, meaning their loans had been transferred to collections after they had failed to make a payment for 270 days or more. Another 3.2 million borrowers were more than 30 days delinquent on their loan payments, and 1.5 million had loans in discretionary forbearance.

For comparison, 17.2 million borrowers were current on their loan payments. But not everyone in current repayment status was actually making progress on their loans. The federal government allows borrowers to make loan payments contingent on their income. While this provides a safety net for low-income borrowers, it also means that loan payments can be insufficient to cover accrued interest. Within five years of entering repayment, 75% of borrowers who choose income-based plans owe more than they originally borrowed. This is a strong signal that these borrowers’ education was not worth the cost.

Loan non-repayment drains resources from federal aid programs. The federal government is expected to lose $19 billion on student loans issued in fiscal year 2022, according to fair-value estimates from the Congressional Budget Office. Loan non-repayment also keeps borrowers in debt for longer and raises the risk of default, which comes with severe financial consequences including wage garnishment and punitive 20% fees.

The federal government is expected to lose $19 billion on student loans issued in fiscal year 2022.

Many prospective students read student loan horror stories in the media and decide that borrowing to attend college is not worth it. But student debt does not have to be a life sentence. Among borrowers who choose standard payment plans and keep up with their payments, over 75% are making progress paying down principal. A quarter of borrowers in this group fully pay off their loans in just four years.

Students who pursue high-value degrees and graduate on time can earn back the cost of their education quickly. Nursing students, for instance, typically enjoy an earnings premium large enough to fully recoup the cost of college within five years. Degrees that yield significant earnings premiums are far more likely to enable students to pay down their loans in a timely manner.

Indeed, a strong correlation emerges between our estimates of ROI and student loan repayment. Student loan repayment is measured as the share of each program’s completers who are current on their federal student loans and have paid down at least $1 of principal within two years, plus the share of borrowers who repaid their loans in full within that time frame. This metric does not capture all potentially positive loan outcomes (for instance, borrowers whose loans are in deferment while they attend graduate school are not counted), but there is nonetheless a clear relationship between repayment and ROI.

The correlation between ROI and student loan repayment holds across all credential types. Bachelor’s-degree programs with negative ROI have an average loan repayment rate of 31%, but BA programs with ROI above $1 million have a repayment rate of 72%. The correlation coefficient between ROI and student loan repayment for bachelor’s degrees is 0.62.

Repayment rates vary widely even at the same institution. University of Arizona students who major in electrical engineering (ROI = $793,000) have a repayment rate of 77%. But their peers who major in anthropology (ROI = $10,000) have a repayment rate of just 38%.

The current state of higher education accountability

The goal of federal financial support for higher education is to benefit students and the economy. But federal grant and loan dollars fund an enormous number of programs with negative ROI.

Existing policy already recognizes that this is a problem, to an extent. The Department of Education calculates a Cohort Default Rate (CDR) for each institution that participates in the federal loan program. The CDR is equal to the share of borrowers who default on their loans within two to three years of entering repayment. If a school’s CDR exceeds 30% for three years in a row, or 40% for one year, the school may lose its access to federal loans. The thinking goes that institutions which don’t provide economic value will experience high loan default rates, and the CDR will catch these.

But in practical terms, the CDR has many problems. For starters, the default rate threshold is set too high. While nearly 5,000 schools participate in the federal loan program, in fiscal year 2018 just eight schools had a cohort default rate high enough to trigger a loss of program eligibility. While hundreds of thousands of borrowers still default every year, only a small fraction attend a school with a high enough default rate to result in sanctions.

The CDR is also vulnerable to manipulation. Some schools pressure borrowers to place their loans into forbearance until after the three-year CDR window closes. The borrowers still default after the forbearance expires, but the default is recorded too late to be captured in CDR. Borrowers in income-based repayment programs who negatively amortize are not captured either. The CDR is also calculated at the institution level, meaning it misses low-quality programs if they are attached to a larger college or university.

Recognizing the shortcomings of the CDR, the Biden administration has signaled its intention to reinstitute the Gainful Employment (GE) rule. Under GE, the Department of Education would terminate federal financial aid eligibility for programs where the ratio of student debt to post-completion earnings is too high. GE improves on CDR by calculating outcomes at the program level and using metrics that are more difficult to manipulate. The major shortcoming is its scope: GE only applies to certificate programs and degree programs at for-profit colleges. More than two-thirds of negative-ROI programs, weighted by student enrollment, would be exempt from GE.

Principles for reform

Members of both political parties recognize that the existing accountability system is insufficient. In recent years, Republicans and Democrats have proposed comprehensive reauthorizations of the Higher Education Act that strengthen accountability rules. While conservatives and liberals can disagree about the proper scope and generosity of federal support for higher education, both sides can agree that the money we do spend should be targeted towards high-value programs.

In an ideal world, the federal government would continue to fund programs that provide a positive return on investment while penalizing programs where ROI is negative. But while ROI is the most comprehensive measure of a program’s financial value, it is difficult and impractical to calculate, particularly on an ongoing basis. Accountability rules for higher education must therefore be based on simpler metrics which can serve as proxies for ROI.

The following seven principles should inform the design of accountability policy.

1. Accountability should be based on outcomes, not inputs. The federal government shouldn’t allocate school funding on the basis of inputs to education such as institutional structure or curriculum. There are many paths to strong ROI, and policymakers should be open to funding programs that achieve good results with nontraditional methods. It follows that accountability rules should consider only whether schools’ outcomes meet the standards we expect of federally-funded programs. As long as schools generate the outcomes we want, policymakers should be agnostic as to how schools produce them.
2. Accountability should consider both benefits and costs. Return on investment in higher education involves both a return (earnings gains) and an investment (tuition and opportunity cost). Without knowing the costs of an investment in education, it is impossible to judge whether the benefits are worth it. For instance, post-graduation earnings of $50,000 represent an excellent outcome for an associate degree but a terrible result for a law program. Holding all programs to the same benefit standard without considering costs will result in an inconsistent accountability regime.
3. Accountability should be sector- and major-neutral. For-profit colleges tend to have poor ROI, while engineering and computer science programs tend to produce excellent financial returns. This leads to a temptation to apply accountability rules only to the for-profit sector, or to provide more generous funding to programs in typically high-ROI fields. But this is a mistake. There are plenty of negative-ROI programs at public and private nonprofit colleges which should be held equally accountable. Likewise, an engineering program shouldn’t be given special treatment simply because it’s an engineering program; all programs should be required to continually prove their financial value.
4. Accountability rules should use graduated penalties rather than termination for enforcement. CDR and GE both use a binary enforcement mechanism: programs are either allowed to continue participating in federal financial aid programs, or they are terminated. But this design fails to create incentives for continuous improvement. ROI is a spectrum. Mediocre programs shouldn’t get a pass simply because they clear some basic threshold. Rather than termination, accountability rules should instead use graduated financial penalties for enforcement. Schools would pay a fee based on how much their outcomes fall short of a given performance benchmark. While the worst schools would find continued participation in federal aid programs financially untenable, most would respond positively to a financial incentive for continuous improvement.
5. Accountability should consider program-level outcomes. Most variation in ROI is due to differences in ROI between fields of study, not between institutions. ROI and other outcomes vary widely between different majors even at the same institution. It follows that considering outcomes at the institution level only, as the CDR does, fails to capture many of the low-value programs that drive poor ROI. One of the simplest ways for an institution to improve its performance is to change the mix of programs it offers. But this will not happen unless the federal government holds institutions accountable for program-level outcomes, or at the very least makes program-level outcomes data available to inform institutions’ decision-making.
6. Accountability rules should not exempt low-ROI but socially valuable programs. Some policymakers might believe that programs which show low or negative ROI are nonetheless worth subsidizing for their social value. Bachelor’s degrees in education (where 44% of programs have negative ROI) are a common example. But these programs should not be exempt from accountability rules. Instead, policymakers should consider directly subsidizing socially valuable professions such as teaching. Topping-up the salaries of teachers would improve education programs’ performance on outcome metrics like loan repayment rates or debt-to-income ratios. But if a program still shows negative ROI after the subsidy, that program likely does not produce social benefits large enough to justify the costs. For this reason, policymakers should continue to hold all programs accountable for their outcomes, even those perceived as socially valuable.
7. Accountability should be proactive rather than reactive. Outcomes-based accountability is superior to inputs-based regulation. But there is a lag of several years between when federal grant and loan dollars are disbursed to institutions and when students’ outcomes are measured. This lag weakens the incentive for good performance and allows unscrupulous institutions to “take the money and run” before penalties kick in. One idea to make accountability policy more proactive is to require institutions to purchase insurance to cover any financial penalties they might incur; insurance companies will provide an extra layer of oversight to ensure that institutions are on track to produce good outcomes.

Accountability is a necessary next step

The federal government funds tens of thousands of higher education programs with low or negative return on investment. Students in negative-ROI programs typically fail to earn back the cost of their education and have more difficulty repaying their student loans. Fortunately, members of both parties recognize that the prevalence of low-value higher education programs is an urgent problem requiring a policy response. The principles described in this report can inform the conversation as policymakers consider how best to structure accountability rules.