College outcomes (part 1): post-graduate salary doesn't tell the whole story
This week, Secretary of Education Arne Duncan announced that the administration will be holding a series of town hall discussions at college campuses around the country on the proposed federal system for college ratings. Spurred by concerns about college affordability, and the general question on whether a college education is worth the substantial financial investment, the new system will rate colleges based on metrics of outcomes, affordability, and accessibility. One proposed metric for comparing institutions is the average salary of graduates at some time point (perhaps 18 months) after graduation. While the intent of this approach (providing a means of measuring determining the economic value of a degree) is understandable, such a metric would be deeply flawed and problematic.
In very general terms, one can paint a very clear picture on the economic value of a college education. According to the latest report by the College Board, in 2011 the median earnings (over the course of a 40-year career of work) of a bachelor’s degree holder are 65% larger than those of a high school graduate. The economic advantages of a college education are also apparent from the unemployment data; in April 2013, unemployment for college graduates was 3.9%, compared to 7.5% for the workforce as a whole. Even in an anemic economy struggling to recover from recession, a college degree adds real value to one’s ability to find a job and to one’s net lifetime earnings.
What makes the current discussions on return on investment different, however, is the use of this metric as a comparative tool: how does the return on investment for graduates from one institution compare to others? Can institutions be ranked, or rated, based on the return on investment metric?
On the surface, the problem sounds very simple – measuring the return on investment for a college education, at least in economic terms, is a relatively straightforward calculation. For each institution, we can determine the costs to students and their families quite well: the total funds spent on tuition and fees, and loan burden carried by families after graduation, and perhaps even wages foregone while studying in school. If we combine this information with data on the average incomes of graduates of the institution, and then compare this to the expected income (without a college degree), and determine the net return (over time) on the initial costs, these data can be compiled and compared institution by institution.
This is the approach taken by the website payscale.com, which publishes an annual college salary report that ranks schools based on average starting salary, average midcareer salary, and a formulation of return on investment. The payscale.com survey is growing in popularity, capturing media attention, attracting critics, and providing fodder for some college and university public relations offices.
In practice, the data and methodology used by the payscale.com rankings are highly flawed. The site does not do a systematic survey of alumni, rather collects data from employers or prospective employees who use other services on the payscale.com site. As a result, the number of alumni actually represented in the survey for any one school is very small; for Kenyon, only 71 alumni records are included (out of the over 15,000 living alumni); for graduates of the past 1 -4 years, only 26 records are included (fewer than 2% of the graduates). Moreover, all employees who have a degree beyond the bachelor’s are excluded (including all who have received professional degrees – JD, MD, MBA – or other graduate degrees). Self-employed, project-based, or contract employees are also excluded. Own a business? You are not included on the survey. Work as an artist, writer, or in another self-employed field? Also excluded (sorry, Kenyon alums Josh Radnor, Allison Janney, John Green, and many others). In other words, the payscale.com rankings are based on an insignificant sampling of graduates, with an arbitrary privileging of some fields over others. Even if one believes the premise that comparisons of colleges in terms of graduate earnings is valid, at best the payscale.com rankings would be a clumsy and misleading application of this approach.
One hopes that the Department of Education will execute this better. But, even if the methodology is improved, the concept itself is fundamentally flawed. Salaries correspond most closely with occupation – typically, engineers are paid more than teachers -- rather than educational institution. One can ask many questions about why this is the case (and, what it means for our society’s values on different occupations and career choices). But, this does make the approach relatively uninformative with respect to differences between schools. Engineering schools (such as Harvey Mudd) will have higher average salaries, not because of their performance as an educational institution, but rather because engineers earn relatively high starting salaries after graduation.
The emphasis on salary and earnings also leaves out the long-term impact that individuals have on society. One may earn a relatively small salary as a teacher, but if one excels at teaching, the total, long-term impact one can have on society is very substantial. The desired outcomes of a Kenyon education include preparation for career success, but also include lifelong engagement as an educated citizen; a life enriched by appreciation and knowledge of the arts, humanities, social and natural sciences; and the ability to creatively apply the skills learned at Kenyon to solve problems, to produce new and original work, and to lead in a variety of walks of life. These are not measured by salary alone.
President Obama himself is an example of someone who chose lower paying positions (as a community organizer, and later as a part-time lecturer in law school , his earnings almost certainly deflated the average graduate earnings of his alma mater, Columbia University). Yet over time he has applied his undergraduate training to broader national service. Job success after graduation is very important, and this factor should not be ignored (either in institutional self-evaluation or by external benchmarks of institutional efficacy). But, to use salary and earnings as the sole proxy for measuring success after completion of an undergraduate education is deeply flawed.