May 20, 2009 the National Association for College Admission Counseling (NACAC) released a study titled, Preparation for College Admission Exams. The media jumped all over the study because they saw an opportunity to blast the test preparation industry and the CollegeBoard. Market Place, an NPR production ran a lead story on the report with the headline, SAT Prep Courses Get Bad Score. The Wall Street Journal ran the following headline, SAT Coaching Found to Boost Scores--Barely. The Boston Globe headline was, Once Mighty SAT Loosing its Clout. I encourage you to read these articles.
There is important information in the report that is not covered in the "dramatic" headlines used by the journalists to grab your attention and sell papers.
The report was commissioned by the National Association for College Admission Counseling (NACAC) and written by Derek C. Briggs, Ph.D. University of Colorado at Boulder. Professor Briggs is the chair of the Research and Methodology Program at the University of Colorado at Boulder. He is an associate professor of quantitative methods and policy analysis. The stated objective of the study is to determine if the, "benefits [of test prep] that can be expected for the typical student outweigh the costs?"(Briggs p.4)
Summary of Test Preparation Research
There are a number of interesting and relevant facts about the research that didn't make the headlines or the body of the articles.
Briggs writes, "although extensive, the academic research base does have limitations."
There are few published studies conducted on students taking admission tests since 2000 and there are no studies on students taking the SAT since changes were made to the test in 2005. In 2005 the CollegeBoard substantially changed the SAT to respond to criticism that the test put some students taking the test at a disadvantage because of social economic background. The CollegeBoard responded by dropping analogies, adding a writing section,eliminating quantitative comparisons and adding algebra II and trig questions to the math section. The research data doesn't include the new SAT test.
Briggs writes, "many previous studies were conducted on small samples or were methodologically flawed." The vast majority of the research doesn't control for the quality, quantity and method of test prep or the characteristics of the student populations. The vast majority of coaching studies conducted over a 40 year period from 1951 and 1991 tended to involve small samples that were not necessarily representative of the national population of high school seniors taking college admissions exams, and of the programs offering test coaching.
In 1992 51% of high school seniors took college admissions tests by 2004 the percentage taking college admissions test increased to over 63% because states began to administer the SAT or ACT to all students.
Study Findings
"The existing academic research base indicates that, on average, test preparation efforts yield a positive but small effect on standardized admissions test scores. Research suggests that the average gains made by coached students are in the neighborhood of 30 points." (Briggs p. 1) Uncoached students will on average improve their test scores by retaking the test therefore the estimate of the effects of coaching is adjusted to account for this effect. (Briggs, 2004) The 30 point gains referenced in Briggs report are for the old SAT, prior to 2005 and is based on a 1600 point scale.
The consensus position according to Briggs is:
Coaching has a positive effect on SAT performance, the magnitude is small although statistically significant.
The effect of coaching is larger on the math section of the exam (10-20 pints) than for the critical reading section (5-10 points).
The Effect of Different Test Preparation Methods
Briggs studied 1990-1992 test takers and 2002-2004 test takers who previously took the PSAT using a nationally representative samples of high school students. Briggs (2002) and Briggs and Dominques (2009)to evaluate the effects of multiple forms of test preparation for the SAT and ACT. The studies concluded that the only methods of test preparation to have a significant effect on SAT-Verbal and SAT-Math score gains were formal modes of coaching:
Private Tutoring or enrollment in a commercial coaching class. The magnitude of the effect was small- SAT-Math (20 points) SAT-Verbal (10 points).
Students who used a computer program had a small but statistically significant negative effect on SAT-Math score of seven points.
No forms of test prep had statistically significant effects on SAT-Verbal scores.
Students who had not previously taken the PSAT, enrollment in a special high school class was associated with a statistically significant negative effect of 10 points on the SAT-Verbal. (Briggs 2002, Briggs and Dominque 2009)
Although the articles published in the Wall Street Journal, Boston Globe, NPR and others call into question the validity of spending money on commercial SAT coaching it is important to ask if your child has the skills and discipline they need to compete for admissions to selective colleges. Would they benefit from the structure and coaching available? Ask about the methodology that the tutor/coach employs and the credentials of the coaches/tutors. Would you want your child to take the most important test of their academic life without preparation? Both the SAT and ACT encourage test takers to prepare.
Your comments are welcome.
No comments:
Post a Comment