Small Gains From College Test Coaching

Coaching for standardized college admission tests yields relatively small average gains of about 30 points overall, according to a study released today that nevertheless urges students to prepare, because even a slight improvement can boost their chances of getting into some schools. The report, commissioned by the nonprofit National Association for College Admission Counseling in Arlington, also urges schools to be cautious about using the SAT and the ACT -- the most widely accepted college entrance exams -- to "make fine-grained distinctions between applicants." "This is important because a 20-point . . . difference between two college applicants could be explained by measurement error, differential access to coaching or both," the report said. "It is strongly recommended that admissions counselors receive training that emphasizes this point." The study, conducted by Derek Briggs, an assistant professor of quantitative methods and policy analysis at the University of Colorado at Boulder, is the broadest and perhaps most authoritative look to date at research on the controversial topic of test preparation. But it reaches a similar conclusion as previous studies, one that critics of test prep courses have long supported. All the studies were conducted when the SAT consisted of verbal and math scores that totaled a possible 1600. A writing section was added in 2005, raising the highest possible score to 2400. Many colleges and universities rely heavily on SAT and ACT scores to help in admissions decisions, and test preparation has become a $4 billion business despite the continuing controversy about whether the scores accurately predict a student's academic abilities. The range of coaching alternatives is vast, from free practice tests to one-on-one tutoring that can cost more than $200 an hour. The latter gives whatever advantage exists to more affluent families. The biggest test prep providers, the Princeton Review and Kaplan Inc., which is owned by the Washington Post Co., offer a host of options. A student can sign up for a three-hour Princeton Review course for less than $100 or spend thousands of dollars on personal tutoring, online or face to face. But Princeton and Kaplan no longer do what some other companies do: guarantee specific outcomes. Although the Web sites of both say they can help raise scores, spokesmen for the companies said they do not guarantee specific results. "It's a complicated issue," said Georgetown University Admissions Dean Charles A. Deacon. "General research says there is little or no effect of test prep, so don't waste your time. However, we do know for some people there is a pretty good effect, and it depends on how much time you spend. And it may help them reach a cutoff point for a scholarship or even admissions at certain state systems." Besides reviewing the existing research on the impact of test preparation, Briggs's study offers original data on how colleges use test scores as part of their admissions decisions. Since 1953, more than 30 studies have tried to evaluate the effectiveness of coaching on the SAT, though most were not representative of the national population of high school seniors. Only two have examined ACT scores. A survey that Briggs developed -- to which 246 colleges and universities responded -- "unexpectedly revealed" that in a substantial minority of cases, colleges reported either that they use a cutoff test score in the admissions process or that a small increase in test scores can have a significant impact on an applicant's chances of admission. A commission convened by the same association late last year reported that many schools misuse test scores, and it recommended, among other things, that schools abandon the practice of using a cutoff score for admissions. The report warned parents to avoid "coaching rip-offs" and urged them to consider less-costly forms of test preparation available from books and the Internet. Spokesmen for the College Board, which administers the SAT, and for ACT Inc. said test preparation is not especially helpful in raising test scores. "The best preparation for the ACT is to take challenging courses in school and to study hard," said ACT spokesman Ed Colby. However, Ned Johnson, president of PrepMatters, a local test prep firm, said coaching can teach students strategies and shortcuts to addressing some questions and lower their anxiety, yielding gains larger than 30 points. "If I consistently only got people a 30-point improvement, it's hard to imagine that savvy consumers would continue investing money for preparation for their kids taking the test," Johnson said.