The Big Test by Nicholas Lemann is a well-written, fascinating, and critical account of the "untold" history of the SAT as the key measure for determining people’s fates in America’s meritocratic society of post-World War Two. Lemann attempts to demonstrate that this is a history of James Bryant Conant’s particular vision of meritocracy, as well as revealing a little luck and a lot of determination on the part of the SAT’s chief entrepreneur, Henry Chauncey. The book has generated a great deal of controversy and debate in the United States (see for instance History of Education Quarterly, Spring 2001), particularly because, in Lemann’s words, "the machinery that Conant and Chauncey and their allies created is today so familiar and all-encompassing that it seems almost like a natural phenomenon, or at least an organism that evolved spontaneously in response to conditions" (p. 6). This book will be of interest to and should be read by anyone researching the history of testing, but its reach is vast and it will also be a valuable resource for those with an interest in American history and politics, as well as entertaining for all readers of this journal. The history is told largely through a great deal of biographical detail woven together to make up a broad narrative. It is broken into three parts. The first centres around the two main figures in the promotion of the SAT, Conant and Chauncey. Conant, president of Harvard University (1933-1953), was a critic of the small group of established elites that had come to dominate the Ivy League universities and American society more generally, chiefly through the inheritance of privilege. Following Thomas Jefferson, Conant wanted to replace this group of powerful elites with one selected on the basis of innate abilities. That is to say, he sought to create a "natural aristocracy" made up of men of "virtue and talent." This group would be selected from all classes in society on the basis of intelligence, and would be trained in the highest educational institutions to be fitted for public service.
Conant had a key role as the head of one of the most prestigious universities in the United States. He first tried out his scheme by selecting and funding a few of the brightest students from the public school system who had been largely excluded from the elite universities by their system of recruitment from private prep schools. Selecting these students through the administration of an aptitude (not an achievement) test proved extremely successful, and Conant pushed for expanding this type of recruitment to all the elite universities. This was the beginning of the SAT and the organization set up to develop, administer, and research these tests, the Education Testing Service (ETS), run by Henry Chauncey. What resulted, however, was not the vision Conant had in mind. Rather, university education was vastly expanded and not limited to the talented few. Moreover, the SAT came to be seen and used as a ticket to the material rewards of society through professional education and self-interest, as opposed to Conant’s vision of a "governing class" dedicated to public service. Instead of providing a generational flux to the social structure based on merit, the SAT replaced one self-interested elite with another, equally self-interested and self-satisfied, and determined to pass on its privileges to its children.
Lemann points out that meritocracy is a contestable concept and object even as it has been made into a common-sense ordering principle of American society. He argues that the de facto method creating this meritocracy, the SAT, has become the axis of this system of placement and distribution with respect to the rewards of society. But the initial intention of the meritocracy designers (Conant and Chauncey) was selection for the limited positions of power, prestige, and influence in the service of society, not selection for self-interested material gain.
According to Lemann, ETS greatly expanded in its first two decades through institutional imperatives, and the skills and personality of Henry Chauncey. For me, this is the richest part of the book. Chauncey is presented as an early believer in the "science" of testing, with an absolute and perhaps naïve faith in the good it can bring to society. Only good could possibly come out of a "social instrument" for rationally sorting the population by ability and guiding individuals into best-fit roles. Chauncey was not so concerned about which particular social vision testing would serve a theme often repeated in this account. The only problem of the SAT as the key selection device was, for Chauncey, its narrow focus on one type of ability: intelligence. He wanted to test all possible abilities, by what he called a "Census of Abilities," in order to best sort and train the American population. He was, however, pragmatic and would worry about the influence of testing later. His first priority was the survival of the new organization, the ETS.
ETS was set up as a private non-profit organization with quasi-public functions of development and research, and yet it was insulated from both taxes and the public by its private, non-profit status. However, in order to support itself it needed commercial income, which came from testing contracts. As Lemann notes:
Because of its need to be self-supporting, and because of Henry Chauncey’s drive and absolute sense of moral mission about his organization, ETS could be counted on to work fiercely to increase the use of its tests…the push would always be to test more rather than to ponder the merits of testing, though that was supposed to be its job The institutional pressures on ETS set up a situation in which first its tests became ubiquitous and consequential, and then the public debate about them took place. In fact, the public debate did not begin for decades. (p. 77)
Lemann attributes the success of ETS and consequently the SAT largely to the unique position of this organization and to deft handling and maneuvrings by Chauncey. The ETS was in a more advantageous position than its rivals in testing: it had an advocate at the top school (Conant); it was a non-profit organization; the war gave it military contracts; there was favourable attention from the media on the "wonders" of testing; and, thanks to the development of machine technologies to mark vast numbers of tests, it was a cheaper option than some of the alternatives in university admissions departments. This is not to mention that implementing this vast system occurred largely outside the public view and debate. This is also where I think Lemann’s argument about Chauncey’s naïveté might be a bit thin. Lemann presents numerous examples of Chauncey’s ability to deflect criticism and to insulate the organization from the outside, of his secretiveness, of the determined effort to produce limited reliability and validity tests (predictability of grades in college six months after the test) rather than more substantive inquiries, and of a good number of side-deals and various public relations exercises. Someone so acutely aware of the criticisms and intent on deflecting them hardly seems naïve.
The third major figure in this book is Clark Kerr, head of the State University of California and chief architect of the "Master Plan" for California’s higher education system, which called for the SAT to sort high school students into a three-tier system of state-sponsored higher education. The plan secured for ETS and the SAT its goal of being the "standard gauge" for admissions across the country, and cemented its status as the top testing agency, in a virtual monopoly position.
The last half of the book takes a different turn. First, it examines the lives of a few of the first generation of meritocrats or "mandarins," those selected for elite schooling by their SAT scores. The introduction of these people provides Lemann with a backdrop against which to explore the tensions and debates around meritocracy and the affirmative-action programs that began in the 1960s. Various new meritocrats then appear as actors in the backroom politics over a critical debate on affirmative action, which is presented in the final section of the book. This style of narrative perhaps underlines the idea that one small group of elites who made the key decisions for society was replaced by another undemocratic group of elites. At the same time it is not completely clear if these people are really representative of a new elite or the dominant class in America. Lemann’s focus seems to be on just the well-intentioned "left liberal" types who were successful yet dissatisfied in their careers (all lawyers) or lives, and with the social system that seemed to benefit them the most. Despite this selectivity, Lemann succeeds in bringing to life this "new mandarin" class.
The history of affirmative action demonstrates the tensions and debates around meritocracy, democracy, and social justice. Affirmative action with respect to higher education was seen as a means of addressing, through various procedures, the "under-performance" of African-Americans and Latinos on SATs. In the last section of the book, Lemann provides a detailed account of the culmination of this debate in the California Civil Rights Initiative or Proposition 209, an anti-affirmative action referendum initiative, which overturned affirmative action in California in the mid-1990s and was seemingly a victory for meritocracy and the SAT.
Lemann should be commended for being critical and not presenting the development of the SAT and meritocracy as a history of the accumulation of knowledge, and the advance of reason and democracy. He brings individual relations, personality, and politics of all sorts to the fore. Nevertheless the book does tend at times towards a focus on the "great men of history": the great visionary, strategist, entrepreneur, implementer, etc. whose personality traits move history. Although this is tempered by presenting nuanced accounts of the negotiations and the broader issues at stake, Lemann is rather selective in presenting these complexities. He seems to slip occasionally into reproducing the goal of the meritocracy individual achievement as the guide to this history and a product of individual personality above other factors like class, business, science, military, and nation-building projects. The book is broad in theme and argument while narrow on causal explanation. Although multi-causal and the result of intersecting interests, some key changes in American society are presented as being brought about through some very narrow interests of the main protagonists. The expansion of SAT is presented like the "frontier logic" Lemann outlines with respect to Conant’s view of education as the new frontier to drive America forward. The SAT and ETS are often presented as pushing through to new frontiers, settling and expanding. Lemann can be criticized for overstating the impact of the SAT and understating the connection to social, economic, and political processes in favour of personality and institutional imperatives.
However, Lemann does a good job of outlining the networks that were created and the infrastructural work that was done in order to make the SAT "common sense" and the "standard gauge." He also highlights the ideological and practical costs of alternatives to the SAT once it was set up. The notion of meritocracy has become a firmly established principle in the United States over and above any collective interest. As the standard measure for a number of qualities, the SAT connected and was invested in by many institutions. There is a certain alignment of a number of communities (testing, research, government, business, education, university, etc.) through the use of this standard gauge and the social and economic demands of competition. Lemann argues that economic costs are a key factor for the growth and maintenance of SAT. Competing with ETS and the SAT is a costly business and alternatives have been found wanting, largely because of costs, but also because of the reliability/validity claims made by ETS. University admissions are an ideal example. As post-secondary education rapidly expanded through state sponsorship and funding, the SAT provided the ideal selecting and sorting tool for thousands of applicants. Admissions are costly, but the SAT places the burden of costs on students. The alternative admission criteria to the SAT are largely excluded from large state universities because of limited budgets and the costs of alternatives (both of which reinforce social-economic factors of exclusion). The SAT might be unique but I think a similar case can also be made for standardized testing and curricula in general as they become the norm in many regions across North America.
This book is valuable for bringing to the forefront a number of key questions concerning schooling in general and, in particular, the increasing use of high-stakes tests. I do not think these questions are adequately dealt with or explored by Lemann’s own conclusions and prescriptions, presented in the "Afterword." The main point repeated throughout the book is that it was a mistake to equate meritocracy with the narrow abilities that are caught by the SAT ("a close relative to IQ") and to make this measure as consequential as possible at such an early point in one’s life. Such a measure excludes a number of seemingly meritocratic virtues like hard work, compassion, creativity, loyalty, initiative, and so on. Furthermore, Lemann critiques Conant’s vision of a "natural aristocracy," arguing that America or any democracy should not have a select elite at all. On the prescriptive side he calls for general improvement across all levels of schooling and for making the university system less hierarchical. He suggests that a new national policy and curriculum, based on some sort of consensus through an open public debate, should be established, with standardized testing on the basis of mastering that curriculum. This prescription seems slightly odd, given Lemann’s exposure to the inherent difficulties of testing and his knowledge of the inner workings of policy development amongst the new elite class that wishes to guard its "earned place." He seems to equate, in this section, sameness or equivalence with equality, and thus he runs into the very problems that he is trying to critique: that is, in the process of creating standards, many skills and abilities will be excluded while others are given more weight ("book learning" for example).
How does this book translate to the Canadian context? We do not have the same history of meritocracy in Canada, but that is not to say we have no conception of "merit"-based sorting, rewards, and criteria. We do not have a single test that has the impact that the SAT has in the U.S. Nevertheless, the conclusions and questions that are raised in this book I think are pertinent to the Canadian context. There has of course been a push for increased testing in Canada (I know mainly of the Ontario experience) and increased weight given to the international regime of testing as a way of assessing system-wide performance. This tendency has also been reflected in an effort to harmonize student achievement or assessment across provinces, largely through standardized testing. The demands for testing have often been supported by claims that they can produce increased accountability in the system and perhaps make schools compete. The tests have been mainly achievement-oriented but are nevertheless narrow in scope some things cannot be measured cheaply in a large-scale test. The idea of an increase in competition among schools and ranking based on performance has also been accompanied by the growth of the school-choice movement, charter schools, religious schools, tax credits, etc. There has also been a move in some provinces, notably Ontario, for a more restrictive standardized curriculum (along the lines of Lemann’s prescription). The following comments can be applied to both the Canadian context and Lemann’s prescriptions.
Lemann claims that reverse engineering to SAT is restrictive and punitive but then seems to support it in a national curriculum. But as noted there are always problems capturing skills via tests. National curricula and standardized tests might also exacerbate effects of student background by ignoring it and privileging a particular body of skills and background while not accounting for the heterogeneity of skills and populations across the country. There has been a movement in some regions over the past decade to tie system performance to standardized tests. This also happens subtly where competition and school choice is introduced and where tests are used as the primary tool for comparison. This is far from introducing measures that would narrow the gap between schools. As a tool for teacher accountability, the test results will reflect the composition of the student body as much as teacher performance. To some extent it takes away teachers’ abilities to use their skills, particularly where teaching to the test has, due to pressure from principals, parents, and others, affected overall school performance. National and centralized curricula often resort to a lowest- common-denominator approach that might result in a disincentive to offer a variety of course options in favour of individual, quantifiable skills and a "back-to-the-basics" curriculum. This also tends to reinforce the importance of the tests and puts pressure on schools to narrow programs and courses and to focus towards the test, which means less emphasis on promoting diverse skills and de-emphasizing co-operative group learning or other innovative programs.
Lemann’s research and his call for a public debate about schooling raise a number of questions. Prop 209 was also an open debate but, as Lemann demonstrates, most of what played out was in backrooms where the new elites (lobbies) and established parties on both sides wrestled each other to control the debate. Who should be awarded the contract this time for constructing the tests? Will a number of curriculum developers be competing for the lucrative contract resulting in another ETS? How will a national curriculum represent differences (regional, ethnic, and so on)? These issues point further to the problem of the precarious nature of institutions like ETS performing quasi-public functions like research and administration while at the same time needing to promote themselves or compete. Is that not the clarion call of a certain rationale of government today smaller government and deregulation, privatization, and competition to produce accountability? Audit and standardization are both sub-industries of this process, subject to competition, attempting to find measures and narrow dollar criteria of assessments. When we deregulate do we not create similar quasi-monopoly situations to Lemann’s ETS?
The book raises but does not explore far enough some important concerns over the power and knowledge relationship with respect to the general inventory that is created by the SAT (or the education system that Lemann advocates). One important consequence is the disciplining of knowledge and possible forms of education and learning to the SAT as the standard measure not just in terms of student assessment but assessment in general of teachers, schools, neighbourhoods, etc. (p. 273). The book also raises a number of interesting questions for further research on the SAT and schooling, such as the politics and technologies of classification, sorting, and the (pedagogical) boundaries created. Lemann does a good job of outlining the ongoing attempts by ETS and others in "black boxing" the science of tests. Creating "hard facts" through limited but compelling reliability and validity tests make the work that goes into creating a standard aptitude score invisible. There is an important line of critique in this book over the SAT as a measure of aptitude or inherent qualities, deemed and promoted as highly reliable (ETS), not to mention as a valid means of assessing ability or merit. Lemann points out that by the time it was clear that the SAT was not a measure of inherent ability (i.e. somewhat unreliable/invalid) the SAT was the dominant admissions device. The consequences were that instead of the SAT factoring out school and background, these variables were known to affect performance. This led to schools being judged on students’ SAT scores, and schools, curriculum, and teachers were encouraged to perform to the test, thus reinforcing its impact.
It is important to look at the power of the notion that we can convert and flatten the diversity of learning, knowledge, and people into a test, number, fact, or quantity. The SAT or standardized tests generally purport simply to measure, but they also produce social objects and knowledge. They produce a measure, kinds of people, potential, curriculum, discourses, ways of seeing and knowing, expertise, and "meritocracy." They create population(s) individualized, assessed, sorted, and classified, and yet homogeneous. They displace or ignore certain kinds of being in the world and certain memberships (in communities, classes, etc.), and impose a new classification with new memberships. This has enormous potential for those wishing to know, assess, and govern a population. Numbers are easily manipulated and when people are tied to and represented by them they become disciplined by these results and the various expert discourses employed to intervene on their behalf. Linking up the SAT (clearly a class- and ethnic-biased enterprise) is useful in disciplining the population (or the education system) to these results but also in making the process seem legitimate, fair, and indisputable. This unbiased, "scientific" sorting process reinforces, for the most part, the structural inequalities in social relations. There are of course exceptions or meritocratic success stories which serve to reinforce this ideology of fairness. I think these are important areas of research for education and in particular for problematizing the standardized testing movement. Nicholas Lemann’s The Big Test is a good starting point for the history of high-stakes testing and this line of inquiry. Despite some of the limitations I have highlighted, this is a thoughtful and well-researched book that draws our attention to some important issues, and generally makes for an enjoyable and informative read.
Paul Giovannetti
Carleton University