Doug Harris and I are seeking a Project Manager for our ongoing study of financial aid, the Wisconsin Scholars Longitudinal Study.
This person will help us with administrative tasks (e.g. handling travel and meeting planning) but also become involved in the daily work of the interviewing portion of the study, and have a chance to learn alot about the research process.
The position is based at the University of Wisconsin-Madison, and you should learn more and apply by clicking here.
Thanks!
Sara
Wednesday, September 16, 2009
Tuesday, September 15, 2009
The Perils of Group-Think
With the President imploring more Americans to go to college and not give up on themselves or their nation, it's understandable that many policymakers want to join in and do their part. Making college more affordable is a very good place to start.
And free tuition-- heck, free anything--sounds great, especially in a recession. Free tuition provided without changes to the state's operating budget or cuts to any other programs, or by digging a deeper deficit, sounds even better. That's probably why Michigan state representative Fred Durhal Jr. thinks he's got a good thing going, proposing to use a new lottery and casino profits to provide free tuition to more than 160,000 students. It's a win-win right? Says Durhal, "You can feel a little better about losing money to the house if you know it's going to go to children."
Except that the money you are losing is being taken from your own children, and given to someone else's. And, since only students who've lived in Michigan continuously for 5 years and earned a 2.5 GPA or better are eligible, the person losing the money is far more likely to be relatively poor, while the kids getting the money are likely to be relatively rich. Let's think-- who moves around a lot and is far less likely to meet that GPA requirement...? Hmmm...
A basic tenet of good social policy-making is targeting. If you want to help students who right now can't afford college, then you need to set the requirements such that they are the (only) ones who are eligible. Not doing so means spreading the wealth among those who don't really need it, diluting the potential impact. And if you really want to help disadvantaged families then don't use a funding mechanism that draws the cash from the program right out of their pockets.
Michigan can look to many other states for better ideas, and for lessons on why this one is a poor one. Take a look at who has benefited from Georgia's lottery-funded HOPE scholarship, which also includes a GPA cutoff. According to expert Don Heller, over 90% of the expenditures have gone to students who would've attended college even without the financial assistance. As long as a scholarship is tied to a GPA cutoff, and family income is correlated with GPAs, then such programs won't effectively reach their intended audience. As long as poor folk dominate those playing the lotto and visiting the casinos, then that funding mechanism doesn't work either.
I'm as pleased as punch that people want to help more poor kids afford college. That's an idea worth glomming on to. But policy proposals require more than good ideas, they require carefully thought out details and a strong theory of action. This one just doesn't cut it.
And free tuition-- heck, free anything--sounds great, especially in a recession. Free tuition provided without changes to the state's operating budget or cuts to any other programs, or by digging a deeper deficit, sounds even better. That's probably why Michigan state representative Fred Durhal Jr. thinks he's got a good thing going, proposing to use a new lottery and casino profits to provide free tuition to more than 160,000 students. It's a win-win right? Says Durhal, "You can feel a little better about losing money to the house if you know it's going to go to children."
Except that the money you are losing is being taken from your own children, and given to someone else's. And, since only students who've lived in Michigan continuously for 5 years and earned a 2.5 GPA or better are eligible, the person losing the money is far more likely to be relatively poor, while the kids getting the money are likely to be relatively rich. Let's think-- who moves around a lot and is far less likely to meet that GPA requirement...? Hmmm...
A basic tenet of good social policy-making is targeting. If you want to help students who right now can't afford college, then you need to set the requirements such that they are the (only) ones who are eligible. Not doing so means spreading the wealth among those who don't really need it, diluting the potential impact. And if you really want to help disadvantaged families then don't use a funding mechanism that draws the cash from the program right out of their pockets.
Michigan can look to many other states for better ideas, and for lessons on why this one is a poor one. Take a look at who has benefited from Georgia's lottery-funded HOPE scholarship, which also includes a GPA cutoff. According to expert Don Heller, over 90% of the expenditures have gone to students who would've attended college even without the financial assistance. As long as a scholarship is tied to a GPA cutoff, and family income is correlated with GPAs, then such programs won't effectively reach their intended audience. As long as poor folk dominate those playing the lotto and visiting the casinos, then that funding mechanism doesn't work either.
I'm as pleased as punch that people want to help more poor kids afford college. That's an idea worth glomming on to. But policy proposals require more than good ideas, they require carefully thought out details and a strong theory of action. This one just doesn't cut it.
Monday, September 14, 2009
Premature Conclusions: More Money, No More Grads?
Some members of the media are covering the release of a new Canadian study, associated with the Educational Policy Institute, that examines the effects of a financial aid program on college-going and completion among low-income students. Researchers at the Measuring the Effectiveness of Study Aid Project tried to isolate those effects by examining what happened following a change in student aid policy in Quebec that increase aid eligibility and decreased reliance on loans. By comparing student outcomes both before and after the policy change, and comparing the outcomes of similar student in Quebec to those in other provinces (where such reforms did not occur), analysts attempted to establish a causal effect of aid.
They conclude that the policy affected access (increasing overall enrollment among students from families making less than $20K per year by 4-6 percentage points), and persistence (increasing retention rates by 6 percentage points) but did not affect graduation rates--at least within the 4-year window of time during which graduation was measured.
While noting that the null findings may stem from that short period of observation, the researcher still goes on record with this conclusion: "These results therefore cast doubt on the efficacy of this reform in particular, and of needs-based grants in general, to improve graduation rates."The headline over at Inside Higher Ed reads "More Money Doesn't Equal More Graduates."
This is a distinctly premature and irresponsible conclusion. First, as one of my graduate assistants James Benson pointed out, "if the percentage of college-eligible students that enrolled in college increased by 5 percent, and the persistence and graduation rates remained entirely static, then the program produced a net gain in the proportion of young adults completing semesters and degrees."
Furthermore, there are many reasons why an effect might not be estimated properly in this study. As my colleagues Doug Harris, Phil Trostel, and I explained in a recent paper, a simple correlation between aid receipt and college success is likely to be negative because students from low-income families, in the absence of aid, are for a variety of reasons less likely to succeed. Unless researchers can convincingly account for all of those reasons – and we argue that very few do – the estimated effects of aid are likely to look smaller than they really are. This study is not very convincing and really doesn't move far beyond a correlation, for many reasons. For example, as another graduate assistant, Robert Kelchen, indicates:
1. The comparison groups (Quebec vs. other provinces) have very different rates of financial aid take-up prior to the reform. This calls the validity of the comparison into question. It's also too bad the researcher didn't see fit to post his tables on the website, since we cannot see whether the differences post-treatment are significant.
2. Quebec saw increases in the enrollment rates of high-income students following the reform, in addition to increases in the enrollment rates of low-income students. If financial aid was the real driver, it shouldn't have affected the (ineligible) high-income students.
These are but a few examples-- if a full research paper (such as would be submitted for academic review) was available, I bet we'd have more concerns.
This is a case of the press jumping the gun and running with a story, and a headline, not supported by the empirical work done by the researchers. We're in a recession, and aid programs cost a lot of money. We do need to know if they work, and in particular if they are cost-effective. But the estimation of impacts should be done more carefully, and results discussed in a much more responsible manner. Sexy but un-informed headlines will do little good-- perhaps even casting a shadow on an effective program, reducing its ability to maintain funding. All of us studying financial aid have an obligation to do much, much better.
They conclude that the policy affected access (increasing overall enrollment among students from families making less than $20K per year by 4-6 percentage points), and persistence (increasing retention rates by 6 percentage points) but did not affect graduation rates--at least within the 4-year window of time during which graduation was measured.
While noting that the null findings may stem from that short period of observation, the researcher still goes on record with this conclusion: "These results therefore cast doubt on the efficacy of this reform in particular, and of needs-based grants in general, to improve graduation rates."The headline over at Inside Higher Ed reads "More Money Doesn't Equal More Graduates."
This is a distinctly premature and irresponsible conclusion. First, as one of my graduate assistants James Benson pointed out, "if the percentage of college-eligible students that enrolled in college increased by 5 percent, and the persistence and graduation rates remained entirely static, then the program produced a net gain in the proportion of young adults completing semesters and degrees."
Furthermore, there are many reasons why an effect might not be estimated properly in this study. As my colleagues Doug Harris, Phil Trostel, and I explained in a recent paper, a simple correlation between aid receipt and college success is likely to be negative because students from low-income families, in the absence of aid, are for a variety of reasons less likely to succeed. Unless researchers can convincingly account for all of those reasons – and we argue that very few do – the estimated effects of aid are likely to look smaller than they really are. This study is not very convincing and really doesn't move far beyond a correlation, for many reasons. For example, as another graduate assistant, Robert Kelchen, indicates:
1. The comparison groups (Quebec vs. other provinces) have very different rates of financial aid take-up prior to the reform. This calls the validity of the comparison into question. It's also too bad the researcher didn't see fit to post his tables on the website, since we cannot see whether the differences post-treatment are significant.
2. Quebec saw increases in the enrollment rates of high-income students following the reform, in addition to increases in the enrollment rates of low-income students. If financial aid was the real driver, it shouldn't have affected the (ineligible) high-income students.
These are but a few examples-- if a full research paper (such as would be submitted for academic review) was available, I bet we'd have more concerns.
This is a case of the press jumping the gun and running with a story, and a headline, not supported by the empirical work done by the researchers. We're in a recession, and aid programs cost a lot of money. We do need to know if they work, and in particular if they are cost-effective. But the estimation of impacts should be done more carefully, and results discussed in a much more responsible manner. Sexy but un-informed headlines will do little good-- perhaps even casting a shadow on an effective program, reducing its ability to maintain funding. All of us studying financial aid have an obligation to do much, much better.
Friday, September 11, 2009
Abandon All Hope (For Reform) Ye Who Enter Here!
At first glance, one might dismiss a recent policy brief authored by a former Bush Administration official as a partisan diatribe against the American Recovery and Reinvestment Act (ARRA) and the Obama Administration. After all, a chief conclusion of the brief authored for the American Enterprise Institute by Andy Smarick (former Deputy Assistant Secretary in the Spellings-era Education Department and in 'W's White House with the Domestic Policy Council), is: "It appears all but certain that the ARRA’s $75 billion in formula-based education programs are a lost cause for education reform. These funds have been used almost exclusively to fill budget holes, and cash-strapped states and districts will likely use what remains of these funds for similar, reform-averse purposes."Abandon all hope (for reform) ye who enter here!
That quoted summary language in the paper *is* perhaps a bit over the top. A "lost cause"? Really? And that's certainly been the takeaway of some blog accounts of this paper (such as this). But that's not really what Smarick is saying nor is it the most important part of this AEI brief. And, as much as he is making that point, his 'lacking in reform' criticism is directed more at the 50 states than at the federal government.
Economic stimulus and a minimization of a short-term funding cliff were among the main aims of ARRA and its education-focused formula dollars. I don't think anyone seriously expected differently. If you read the ARRA web page, it largely spells this out. Now, the Education Department did envision that State Fiscal Stabilization Funding would be used to promote reform as well, and despite an initial look by the GAO, some dollars may accomplish reform, but how on earth could there yet be any real evidence of reform let alone impact when the 2009-10 school year has just begun in most places?!? In addition, as Smarick notes, the economic downturn and its effect on state budgets was far worse than anticipated at the time that ARRA was enacted in early 2009, which lessened the likelihood of these dollars doing anything less than filling holes.
Smarick's take on the competitive aspects of ARRA -- the Race to the Top and the Investing in Innovation (I3) funds -- is generally fair and balanced. He raises important questions about the general risks to any reform push, and specifically to ARRA. Smarick identifies several factors that may reduce the likelihood that competitive dollars will further education reform: on-going state budgetary challenges, resistance to specific reform components, and lack of faithful and vigorous implementation. He warns of "Trojan horse" applications where states will seek the money, but won't use it for reform. Of course, unmentioned are a whole host of other potential roadblocks, such as resistance from school districts, lack of buy-in from teachers and school administrators, lack of capacity to implement reforms, consultants and subcontractors who can't deliver promised expertise or technical assistance, data systems that cannot accurately match student and teacher data, etc.
Read the brief. Or check out a summary at Flypaper.
Thursday, September 10, 2009
2-Year Proprietaries: Higher Graduation Rates At What Cost?
For several years researchers have debated the relative merits of public community colleges vs. 2 year for-profit schools, in part because the latter are deemed a relevant comparison group for the former-- they share some of the same students, offer the same kinds of degrees, etc. These comparisons have become part of the basis for assessing whether there is a "community college penalty" -- a negative effect of choosing that kind of school over an alternative. The basic numbers certainly seem damning. For example, according to a study by Ann Person and Jim Rosenbaum, among students starting college as part of the Beginning Postsecondary Study in 1995-1996, 42% of those attending a 2-year proprietary completed an associate's degree or higher within 3 years, compared to just 8% of those who began in community colleges.
Of course, those differences in outcomes could be attributable to many things aside from institutional practices--and therefore the same authors have also tested for differences using propensity score matching, which attempts to generate an apples-apples comparison based on observable characteristics of students and colleges. In a recent article they find that while the students attending public and private 2-year colleges overlap substantially in their attributes, their degree outcomes differ. Students are about 14% more likely to complete a degree of any kind if they attend a private rather than public 2-year college (this impact is significant only for students who typically attend a community college--not for those who typically attend a proprietary).
What explains these differences? In a recent book, Jim Rosenbaum and his colleagues rely on largely qualitative data from Chicago colleges to argue that community colleges have a lot to learn from 2-year for-profits, in particular when it comes to advising and job placement services. For example, they contend that for-profits structure students' experience in ways that help them overcome gaps in their "college knowledge," such as those resulting from not having college-educated parents. They also note, however, that it's also possible that academic standards are lower at private colleges, and admit that there's no evidence that employers treat the degrees earned at the two types of colleges any differently. Moreover, there are no discernable differences in earnings based on the type of 2-year college attended.
Now a new report from the College Board adds an interesting wrinkle to the story. Ok, so it's possible that students are more likely to graduate from career colleges. But they graduate with a lot more debt. Data from the 2007-2008 National Postsecondary Student Aid Study reveal that 61% of community college students graduate with less than $10,000 in debt, compared to only 22% of students graduating from 2 year for-profits. In contrast, 19% of graduates from 2 year for-profits have $30,000 or more in student loans, compared to only 5% of community college graduates. Nearly all students (98%) finishing at 2 year for-profit colleges have taken on a loan, compared to just 38% of community college graduates.
Student debt has consequences for later decisions, including choice of occupation, ability to secure a home mortgage, start a family, etc. These latest statistics therefore lead to an important question: is the increased probability of college graduation observed among 2 year for-profit students offset by their higher levels of debt upon finishing? Are the two data points related-- e.g. are community college students with more debt less likely than comparable 2 year for-profit students to finish college, meaning that the College Board's comparisons are skewed? This is possible, if we believe that stronger advising at for-profits helps keep students enrolled and/or encourages them to take on debt and work less, promoting persistence. In any case, while we await answers to these questions, we might want to rethink the tendency to tout the for-profits as a model to which community colleges ought to aspire.
Of course, those differences in outcomes could be attributable to many things aside from institutional practices--and therefore the same authors have also tested for differences using propensity score matching, which attempts to generate an apples-apples comparison based on observable characteristics of students and colleges. In a recent article they find that while the students attending public and private 2-year colleges overlap substantially in their attributes, their degree outcomes differ. Students are about 14% more likely to complete a degree of any kind if they attend a private rather than public 2-year college (this impact is significant only for students who typically attend a community college--not for those who typically attend a proprietary).
What explains these differences? In a recent book, Jim Rosenbaum and his colleagues rely on largely qualitative data from Chicago colleges to argue that community colleges have a lot to learn from 2-year for-profits, in particular when it comes to advising and job placement services. For example, they contend that for-profits structure students' experience in ways that help them overcome gaps in their "college knowledge," such as those resulting from not having college-educated parents. They also note, however, that it's also possible that academic standards are lower at private colleges, and admit that there's no evidence that employers treat the degrees earned at the two types of colleges any differently. Moreover, there are no discernable differences in earnings based on the type of 2-year college attended.
Now a new report from the College Board adds an interesting wrinkle to the story. Ok, so it's possible that students are more likely to graduate from career colleges. But they graduate with a lot more debt. Data from the 2007-2008 National Postsecondary Student Aid Study reveal that 61% of community college students graduate with less than $10,000 in debt, compared to only 22% of students graduating from 2 year for-profits. In contrast, 19% of graduates from 2 year for-profits have $30,000 or more in student loans, compared to only 5% of community college graduates. Nearly all students (98%) finishing at 2 year for-profit colleges have taken on a loan, compared to just 38% of community college graduates.
Student debt has consequences for later decisions, including choice of occupation, ability to secure a home mortgage, start a family, etc. These latest statistics therefore lead to an important question: is the increased probability of college graduation observed among 2 year for-profit students offset by their higher levels of debt upon finishing? Are the two data points related-- e.g. are community college students with more debt less likely than comparable 2 year for-profit students to finish college, meaning that the College Board's comparisons are skewed? This is possible, if we believe that stronger advising at for-profits helps keep students enrolled and/or encourages them to take on debt and work less, promoting persistence. In any case, while we await answers to these questions, we might want to rethink the tendency to tout the for-profits as a model to which community colleges ought to aspire.
Where Have You Been?
A spate of recent articles, including those covering Bill Bowen and Mike McPherson's new book (which I promise to review just as soon as my copy arrives), have left me a bit perplexed-- wondering aloud "where have you all been?" The punchline each time is that a fair proportion of adults starting college are not finishing. Yes, and duh. This is not new, and if it's news well I guess it's only because we've deliberately kept our heads in the sand.But there's no way that folks like New York Times reporter David Leonhardt have been deliberately oblivious, and yet he's writing about low college completion rates as if they've just been unearthed. In a recent blog post, Kevin Carey implied the same-- just as he did in a recent American Enterprise Institute report. But this has been a prominent topic of discussion for years--maybe a decade plus! Just look at Kevin's own 2004 report A Matter of Degrees (which received plenty of media coverage), or the Spellings Commission report, or Claudia Goldin and Larry Katz's book. I know I could go back several more years and find plenty more evidence.
I think it's one thing to imply something is new when it isn't (because again, maybe you just didn't know, or you feel the issue still is widely known enough and want to beat the drum more), and it's another thing entirely to claim that policymakers still aren't paying attention. In Leonhardt's case, he's simply wrong when he says the current Administration isn't focused on college completion. Um, how about that $2.5 billion Access and Completion Fund, part of Obama's original budget proposal? What about the performance (outcomes)-based components of the new community college monies contained in HR 3221? Foundations like Lumina and Gates have been beating this drum for years, and those in the Administration are well aware. No one in DC is saying institutions should continue to be judged solely based on enrollment (even enrollment of disadvantaged groups). There is plenty of ado about completion rates. The question is now, what exactly are the best solutions? That's a debate that needs to be richer and more visible, since the answers are far from clear-- and we'd be terribly wrong to simply resort to NCLB-style responses that remind me of my toddler: "Institutions bad. Do wrong. I punish you and you do better. Now." Let's direct our energies toward really identifying the sources of the problems, and developing a sense of how reforms can be most effective. When I get a chance to read the new Bowen and McPherson book, I'm hoping I come away with new ideas on how to do that.
Sunday, September 6, 2009
Image is Everything
Sunday's New York Times features a Style section article that quite frankly turned my stomach (at least, I'm pretty sure it was the article and not the 6 month old fetus I'm carrying!). It describes a debate over Harvard's decision to sign on to a new, expensive preppy clothing line-- one that charges more than $150 for a shirt, and up to $500 for a sports coat. A variety of opinions are represented, from that of the director of admissions and financial aid ( a former aid recipient himself) to an undergraduate who said, “I think it’s good that it’s [Harvard's] doing something to make money."These deals apparently generate about $500,000 per year for the university, which (poor baby) saw its endowment decline by 30% last year. And that money goes to financial aid, so we're not supposed to worry that Harvard's being greedy.
And that's the main issue the reporter tackles--whether the decision to say yes to a clothing line that portrays an elite undergraduate student body conflicts with Harvard's stated goals of expanding diversity. Whether the money raised is enough to cover the additional costs associated with outreach. The "damage" done.
Well, of course it's not! Image, we all know, is everything-- especially when it comes to those families who rely on media for information in the absence of more informed sources. Harvard's biggest obstacles to bringing in more students from disadvantaged backgrounds are: (1) image; (2) cost of attendance; and (3) admissions requirements. The school is trying to conquer the second one with financial aid, by promising to cover all demonstrated need. That sounds great, but the fact is that the number of admitted students with tremendous financial need isn't very substantial-- if it were, the amount of money required to fulfill that promise would be much more foreboding.
The really poor kids just aren't applying in large numbers to Harvard and that won't change unless it becomes a place that doesn't scream "money, money, money" so loudly to everyone who's ever heard of it. The message that "aid is available, costs are covered" is a good one. But it doesn't neatly translate to "I'll be able to afford to go and enjoy myself and fit in with these kids."
Will TV commercials and print ads featuring Harvard blue bloods generate enough revenue to pay for some more scholarships? Definitely. Will that even begin to offset the damage done by further demonstrations of the internal inconsistencies and contradictions associated with a place that simultaneously wants to do good and yet be the very best? Is anyone seeing those ads (or not seeing them) buying that Harvard's really now open to kids wearing WalMart t-shirts? No way.
Subscribe to:
Posts (Atom)