Atkinson, Richard C., “Achievement versus Aptitude in College Admissions,” Issues in Science and Technology, Winter 2001–02, pp. 31–36. Copyright c???? 2002 by the University of Texas at Dallas, Richardson, Texas. Reprinted with permission.

Geiser, Saul with Roger Studley, “UC and the SAT: Predictive Validity and Differential Impact of the SAT I and SAT II at the University of California,” Educational Assessment, vol. 8, no. 1, pp. 1–26. Copyright c???? 2002 by Lawrence Erlbaum Associates, Inc. Reprinted with permission.

Lawrence, Ida M. et al., “A Historical Perspective on the Content of the SAT R????,” College Board Report No. 2003-3. Copyright c???? 2003 by College Entrance Examination Board. Adapted with permission. All rights reserved. www.collegeboard.com.

Published in 2004 by
Routledge
29 West 35th Street
New York, NY 10001
www.routledge-ny.com Published in Great Britain by
Routledge
11 New Fetter Lane
London EC4P 4EE
www.routledge.co.uk

Copyright C???? 2004 by Taylor and Francis Books, Inc.

Routledge is an imprint of the Taylor and Francis Group.

This edition published in the Taylor & Francis e-Library, 2004.

Cataloging-in-Publication Data is available from the Library of Congress

ISBN 0-203-46393-5 Master e-book

ISBN ISBN 0-203-47089-3 (Adobe eReader Format)

ISBN 0-415-948355 (paperback)
ISBN 0-415-948347 (hardcover)

Achievement versus Aptitude in College Admissions

RICHARD C. ATKINSON

Every year, more than a million high school students stake their futures on the nation’s most widely used admissions test, the SAT I. Long viewed as the gold standard for ensuring student quality, the SAT I has also been considered a great equalizer in U.S. higher education. Unlike achievement tests such as the SAT II, which assess mastery of specific subjects, the SAT I is an aptitude test that focuses on measuring verbal and mathematical abilities independent of specific courses or high school curricula. It is therefore a valuable tool, the argument goes, for correcting the effects of grade inflation and the wildly varying quality of U.S. high schools. And it presumably offers a way of identifying talented students who otherwise might not meet traditional admissions criteria, especially high-potential students in low-performing high schools.

In February 2001, at the annual meeting of the American Council on Education (ACE), I delivered an address questioning the conventional wis-dom about the SAT I and announced that I had asked the Academic Senate of the University of California (UC) to consider eliminating it as a require-ment for admission to UC. I was unprepared for the intense public reaction to my remarks. The day before I was scheduled to deliver them, I went to the lobby of my hotel to get a copy of the Washington Post.Iwasastoundedto find myself and excerpts from the speech on the front page; an early version had been leaked to the press. To my further astonishment, an even more detailed story appeared on the front page of the New York Times.

And that was only the beginning. In the months since my address, I have heard from hundreds of college and university presidents, CEOs, alumni, superintendents, principals, teachers, parents, students, and many others from all walks of life. Television programs, newspaper editorials, and maga-zine articles have presented arguments pro and con. I was most struck by the Time magazine article that had a picture of President Bush and me side by side. The headline read, “What do these two men have in common?” Those who have speculated that the answer is that we had the same SAT scores are wrong. I did not take the SAT. I was an undergraduate at the University of Chicago, and at that time the university was adamantly opposed to the con-cept of aptitude tests and used achievement tests in its admissions process. Time was simply observing that we share an interest in testing.

It came as no surprise that my proposal to take a hard look at the role and purpose of the SAT I and standardized tests in general attracted the attention of educators, admissions officers, and testing experts. I have been impressed and pleased by the many researchers, professors, and psychometricians who have shared with me their findings and experience regarding the SAT. But I was also surprised at the number of letters I received from people who had no professional connection with higher education. I heard from a young woman—an honors graduate of UC Berkeley with an advanced degree from Princeton—who had been questioned about her 10-year-old SAT scores in ajobinterview;anattorneywho,despitedecadesofsuccess,stillremembers the sting of a less-than-brilliant SAT score; an engineer who excelled on the SAT but found it bore no relation to the demands of college and his profession; a science student who scored poorly on the SAT and was not admitted to his college of choice but was elected to the National Academy of Sciences in later years. Clearly, the SAT strikes a deep chord in the national psyche.

The second surprise in the months after my speech was the degree of confusion about what I proposed and why I proposed it. For example, some people assumed I wanted to eliminate the SAT I as an end run around Proposition 209, the 1996 California law banning affirmative action. That was not my purpose; my opposition to the SAT I predates Proposition 209 by many years. And as I said in my ACE speech, I do not anticipate that ending the SAT I requirement by itself would appreciably change the ethnic or racial composition of the student body admitted to UC.

Others assumed that because I am against the SAT I, I am against stan-dardized tests in general. I am not; quite the opposite is true. Grading prac-tices vary across teachers and high schools, and standardized tests provide a measure of a student’s achievements that is independent o fgrades.

But we need to be exceedingly careful about the standardized tests we choose.

So much for what I did not propose. Let me turn briefly to what I did pro-pose. I requested the Academic Senate of UC to consider two further changes in addition to making the SAT I optional. The first is to use an expanded set of SAT II tests or other curriculum-based tests that measure achieve-ment in specific subject areas until more appropriate tests are developed. The second is to move all UC campuses away from admissions processes employing quantitative formulas and toward a comprehensive evaluation of applicants.

In a democratic society, I argued, admitting students to a college or uni-versity should be based on three principles. First, students should be judged on the basis of their actual achievements, not on ill-defined notions of ap-titude. Second, standardized tests should have a demonstrable relationship to the specific subjects taught in high school, so that students can use the tests to assess their mastery of those subjects. Third, U.S. universities should employ admissions processes that look at individual applicants in their full complexity and take special pains to ensure that standardized tests are used properly in admissions decisions. I’d like to discuss each in turn.

Aptitude versus Achievement

Aptitude tests such as the SAT I have a historical tie to the concept of innate mental abilities and the belief that such abilities can be defined and mean-ingfully measured. Neither notion has been supported by modern research. Few scientists who have considered these matters seriously would argue that aptitude tests such as the SAT I provide a true measure of intellectual abilities.

Nonetheless, the SAT I is widely regarded as a test of basic mental ability that can give us a picture of students’ academic promise. Those who support it do so in the belief that it helps guarantee that the students admitted to college will be highly qualified. The SAT I’s claim to be the “gold standard of quality” derives from its purported ability to predict how students will perform in their first year of college.

Nearly 40 years ago, UC faculty serving on the Academic Senate’s Board of Admissions and Relations with Schools (BOARS) gathered on the Santa Barbara campus to consider the merits of the SAT and achievement tests. At that point, UC had only run experiments with both kinds of tests. In the actual process of admissions, UC used standardized tests in admissions decisions for only a small percentage of students who did not qualify on the basis of their grades in selected courses. BOARS wanted answers to a couple of critical questions: What is the predictive power—what researchers call the “predictive validity”—of the SAT for academic success at UC? How might it improve the process of admissions?

To answer these questions, BOARS launched a study that compared the SAT and achievement tests as predictors of student performance. The results were mixed. In the view of the board, the achievement tests proved a more useful predictor of student success than did the SAT, both in combination with grades and as a single indicator. But the benefits of both tests appeared marginal at the time. As a result, both the SAT and achievement tests re-mained largely an alternative method for attaining UC eligibility. In 1968, UC began requiring the SAT I and three SAT II achievement tests, although applicants’ scores were not considered in the admissions process. Rather, the SAT I and SAT II tests remained largely a way of admitting promising students whose grades fell below the UC standard and an analytical tool to study the success patterns of students admitted strictly by their grades in UC-required courses.

This policy lasted until the late 1970s. As historian John Douglass has noted in a number of studies on the history of UC admissions, not until 1979 did the university adopt the SAT as a substantial and formal part of the regular admissions process. That year, BOARS established UC’s current Eligibility Index: a sliding scale combining grade point average (GPA) in required courses with SAT scores to determine UC eligibility. Even then, GPA remained the dominant factor in this determination. UC established the Eligibility Index largely as a way of reducing its eligibility pool in light of a series of studies that showed UC accepting students well beyond its mandated top 12.5 percent of statewide graduates. The decision to include SAT scores in the Eligibility Index was based not on an analysis of the SAT’s predictive power but on its ability to serve as a screen that would reduce the pool of eligible students.

Fortunately, today we do have an analysis of the SAT’s value in admissions decisions. Because our students have been taking the SAT I and the SAT II for more than three decades, UC is perhaps the only university in the country that has a database large enough to compare the predictive power of the SAT Iwiththatoftheachievement-basedSATIItests.UCresearchersSaulGeiser and Roger Studley have analyzed the records of almost 78,000 freshmen who entered UC over the past four years. They concluded that the SAT II is, in fact, a better predictor of college grades than the SAT I. The UC data show that high school grades plus the SAT II account for about 21 percent of the explained variance in first-year college grades. When the SAT I is added to high school grades and the SAT II, the explained variance increases from 21 percent to 21.1 percent, a trivial increment.

Our data indicate that the predictive validity of the SAT II is much less affected by differences in socioeconomic background than is the SAT I.

After controlling for family income and parents’ education, the predictive power of the SAT II is undiminished, whereas the relationship between SAT I scores and UC freshman grades virtually disappears. These findings suggest that the SAT II is not only a better predictor but also a fairer test for use in college admissions, because its predictive validity is much less sensitive than is the SAT I to differences in students’ socioeconomic background. Contrary to the notion that aptitude tests are superior to achievement tests in identifying high-potential students in low-performing schools, our data show the opposite: The SAT II achievement tests predict success at UC better than the SAT I for students from all schools in California, including the most disadvantaged.

UC data yield another significant result. Of the various tests that make up the SAT I aptitude and the SAT II achievement tests, the best single predictor of student performance turned out to be the SAT II writing test. This test is the only one of the group that requires students to write something in addition to answering multiple-choice items. Given the importance of writing ability at the college level, it should not be surprising that a test of actual writing skills correlates strongly with freshman grades.

When I gave my speech to ACE, this comprehensive analysis of the UC data comparing the two tests was not available. My arguments against the SAT I were based not on predictive validity but on pedagogical and philo-sophical convictions about achievement, merit, and opportunity in a demo-cratic society. In my judgment, those considerations remain the most telling arguments against the SAT I. But these findings about the predictive validity of the SAT I versus the SAT II are stunning.

Curriculum-Based Tests

If we do not use aptitude tests such as the SAT I, how can we get an accurate picture of students’ abilities that is independent of high school grades? In my view, the choice is clear: We should use standardized tests that have a demonstrable relationship to the specific subjects taught in high schools. This would benefit students, because much time is currently wasted inside and outside the classroom prepping students for the SAT I; the time could be better spent learning history or geometry. And it would benefit schools, because achievement-based tests tied to the curriculum are much more attuned to current efforts to improve the desperate situation of the nation’s K-12 schools.

One of the clear lessons of U.S. history is that colleges and universities, through their admissions requirements, strongly influence what is taught in the K-12 schools. To qualify for admission to UC, high-school students must attain specified grades in a set of college-preparatory classes that includes mathematics, English, foreign languages, laboratory sciences, so-cial sciences, and the arts. These requirements let schools and students alike know that we expect UC applicants to have taken academically challenging courses that involve substantial reading and writing, problem-solving and laboratory work, and analytical thinking, as well as the acquisition of factual information. These required courses shape the high-school curriculum in direct and powerful ways, and so do the standardized admissions tests that are also part of qualifying for UC.

Because of its influence on K-12 education, UC has a responsibility to articulate a clear rationale for its test requirements. In my ACE address in February, I suggested what that rationale might contain: 1) The academic competencies to be tested should be clearly defined; in other words, testing should be directly related to the required college preparatory curriculum. 2) Students from any comprehensive high school in California should be able to score well if they master the curriculum. 3) Students should be able, on reviewing their test scores, to understand where they did well or fell short and what they must do to earn higher scores in the future. 4) Test scores should help admissions officers evaluate the applicant’s readiness for college-level work. The Board of Admissions and Relations with Schools is in the process of developing principles to govern the selection and use of standardized tests. These principles will be an extremely important contribution to the national debate about testing.

Universities in every state influence what high schools teach and what students learn. We can use this influence to reinforce current national efforts to improve the performance of U.S. public schools. These reform efforts are based on three principal tenets: Curriculum standards should be clearly defined, students should be held to those standards, and standardized tests should be used to assess whether the standards have been met.

The SAT I sends a confusing message to students, teachers, and schools. It says that students will be tested on material that is unrelated to what they study in their classes. It says that the grades they achieve can be devalued by a test that is not part of their school curriculum. Most important, the SAT I scores only tell a student that he or she scored higher or lower than his or her classmates. They provide neither students nor schools with a basis for self-assessment or improvement.

Appropriate Role of Standardized Tests

Finally, I have argued that U.S. universities should employ admissions pro-cesses that look at individual applicants broadly and take special pains to ensure that standardized tests are used properly in admissions decisions. Let me explain this statement in terms of UC.

UC’s admissions policies and practices have been in the spotlight of public attention in recent years as California’s diverse population has expanded and demand for higher education has skyrocketed. Many of UC’s 10 campuses receive far more applicants than they can accept. Thus, the approach we use to admit students must be demonstrably inclusive and fair.

To do this, we must assess students in their full complexity. This means considering not only grades and test scores but also what students have made of their opportunities to learn, the obstacles they have overcome, and the special talents they possess. To move the university in this direction, I have made four admissions proposals in recent years:

Eligibility in the Local Context (ELC), or the Four Percent Plan, grants UC eligibility to students in the top 4 percent of their high school graduating class who also have completed UC’s required college preparatory courses. Almost 97 percent of California public high schools participated in ELC in its first year, and many of these had in the past sent few or no students to UC.

Under the Dual Admissions Program approved by the regents in July 2001, students who fall below the top 4 percent but within the top 12.5 percent of their high school graduating class would be admitted simultaneously to a community college and to UC, with the proviso that they must fulfill their freshman and sophomore requirements at a community college (with asolidGPA) before transferring to a UC campus. State budget difficulties have delayed implementation of the Dual Admissions Program, but we hope to launch it next year.

For some years, UC policy has defined two tiers for admission. In the first tier, 50 to 75 percent of students are admitted by a formula that places principal weight on grades and test scores; in the second tier, students are assessed on a range of supplemental criteria (for example, difficulty of the courses taken, evidence of leadership, or persistence in the face of obstacles) in addition to quantitative measures. Selective private and public universities have long used this type of comprehensive review of a student’s full record in making admissions decisions. Given the intense competition for places at UC, I have urged that we follow their lead. The regents recently approved the comprehensive review proposal, and it will be effective for students admitted in fall 2002.

Finally, for the reasons I have discussed above, I have proposed that UC make the SAT I optional and move toward curriculum-based achievement tests. The Academic Senate is currently considering this issue, and its review will likely be finished in spring 2002, after which the proposal will go to the Board of Regents.

The purpose of these changes is to see that UC casts its net widely to iden-tify merit in all its forms. The trend toward broader assessment of student talent and potential has focused attention on the validity of standardized tests and how they are used in the admissions process. All UC campuses have taken steps in recent years to ensure that test scores are used properly in such reviews; that is, that they help us select students who are highly qualified for UC’s challenging academic environment. It is not enough, however, to make sure that test scores are simply one of several criteria considered; we must also make sure that the tests we require reflect UC’s mission and purpose, which is to educate the state’s most talented students and make educational opportunity available to young people from every background.

Achievement tests are fairer to students because they measure accom-plishment rather than ill-defined notions of aptitude; they can be used to improve performance; they are less vulnerable to charges of cultural or so-cioeconomic bias; and they are more appropriate for schools, because they set clear curricular guidelines and clarify what is important for students to learn. Most important, they tell students that a college education is within the reach of anyone with the talent and determination to succeed.

For all of these reasons, the movement away from aptitude tests toward achievement tests is an appropriate step for U.S. students, schools, and uni-versities. Our goal in setting admissions requirements should be to reward excellence in all its forms and to minimize, to the greatest extent possible, the barriers students face in realizing their potential. We intend to honor both the ideal of merit and the ideal of broad educational opportunity. These twin ideals are deeply woven into the fabric of higher education in this country. It is no exaggeration to say that they are the defining characteristics of the U.S. system of higher education.

The irony of the SAT I is that it began as an effort to move higher education closer to egalitarian values. Yet its roots are in a very different tradition: the IQ testing that took place during the First World War, when two million men were tested and assigned an IQ based on the results. The framers of these tests assumed that intelligence was a unitary inherited attribute, that it was not subject to change over a lifetime, and that it could be measured and individuals could be ranked and assigned their place in society accordingly. Although the SAT I is more sophisticated from a psychometric standpoint, it evolved from the same questionable assumptions about human talent and potential.

The tests we use to judge our students influence many lives, sometimes profoundly. We need a national discussion on standardized testing, informed by principle and disciplined by empirical evidence. We will never devise the perfect test: a test that accurately assesses students irrespective of parental education and income, the quality of local schools, and the kind of commu-nity students live in. But we can do better. We can do much better.

References

Atkinson, R. C. (2001). “Standardized tests and access to American universities,” 2001 Robert

Atwell Distinguished Lecture, 83rd Annual Meeting of the American Council on Education, Washington, D.C., February 18, 2001. Online at http://www.ucop.edu/pres/prespeeches. html.

Douglass, J. A. (1997). Setting the conditions of admissions: The role of university of California faculty in policymaking. Study commissioned by the University of California Academic Senate, February 1997. Online at http://ishi.lib.berkeley.edu/cshe/jdouglass/publications.html. Douglass, J. A. (2001). “Anatomy of conflict: The making and unmaking of affirmative action at theUniversity of California,” in D. Skrentny (Ed.), Color Lines: Affirmative Action, Immigration and Civil Rights Options for America. Chicago: University of Chicago Press.

Geiser, S. and Studley, R. (2001). UC and the SAT: Predictive validity and differential impact of the SAT I and SAT II at the University of California. University of California Office of the President, October 29, 2001. Online at http://www.ucop.edu/pres/welcome.html.