Finding the Right Accommodations for Assessing English Language Learners
Sarah Finck, New York University, sbf248@nyu.edu
Accommodations have been introduced in order to help make English Language Learners’ (ELLs) performance on standardized tests a more accurate indication of their content knowledge and less a reflection of their English ability. This article discusses two types of accommodations, linguistic simplification and dictionary use, and their differential effect for subgroups of ELLs.
Standardized testing is a reality for students in the United States today whether they have achieved English proficiency or not. Prior to the No Child Left Behind legislation of 2001, which mandated the testing of all students in order to hold schools and districts accountable for making adequate yearly progress toward high educational standards, English language learners were often excluded from large-scale testing. Now that exemption from testing is no longer an option, testing accommodations have been introduced to help make English language learners’ performance on standardized tests a more accurate indication of their content knowledge and less a reflection of their English ability.
Coming in many shapes and sizes, accommodations are “support provided students for a given testing event, either through modification of the test itself or through modification of the testing procedure, to help students access the content in English and better demonstrate what they know” (Butler & Stevens, 1997, p. 5). The most commonly researched accommodations include linguistic modification or simplification; glossaries and customized dictionaries; published dictionaries; dual-language test booklets; translated tests; and oral administration. Yet even with a constantly growing body of research on the subject, it is becoming clear that we still do not know enough about which accommodations are most effective for which test-takers; in particular, it appears that proper assignment of accommodations may be a function of each student’s individual language and background characteristics.
This article discusses two types of accommodations, linguistic simplification and dictionary use, which are among the most researched accommodations for English language learners (Abedi & Gándara, 2006; Abedi, Hofstetter, & Lord, 2004), as well as the differential effect of these accommodations for different subgroups of English language learners. The article concludes by proposing future research needed to further our understanding of how English language learners interact with tests and how accommodations can help improve the validity of the interpretations that are made on the basis of their test performance.
Linguistic Simplification
Finding a way to level the playing field between English language learners and non-English language learners is challenging, but linguistic simplification has proven to be an effective accommodation for ensuring that the tests measure what they set out to measure, namely, content area knowledge. Research has shown that the linguistic complexity of test items can be a large contributing factor to the performance gap between English language learners and non-English language learners on assessments of math and science knowledge (Abedi, Lord, & Plummer, 1997); in particular, long noun phrases, passive voice constructions, long question phrases, prepositional phrases, subordinate clauses, conditional clauses, relative clauses, abstract nouns, and negation (Abedi et al., 1997; Rivera & Stansfield, 2004) are problematic for English learners who have lower syntactic awareness skills than do non-English language learners (Abedi & Gándara, 2006). Consequently, linguistic simplification, or modification, of test items has been identified as a solution to this problem of complexity. By systematically rewriting instructions and questions so as to eliminate or change the above structures and make the language more straightforward, construct irrelevant variance is reduced and English language learners perform better while non-English language learners remain relatively unaffected.
Several studies provide support for the validity of linguistic modification, and they lead us to see that the effectiveness of the accommodation depends on a handful of factors. Abedi, Lord, Hofstetter, and Baker (2000) looked at four different accommodation strategies (linguistic simplification of test items; glossary; extra time; and glossary plus extra time) and their effect on the National Assessment of Educational Progress (NAEP) Mathematics test for eighth-grade students in southern California. Though the glossary plus extra time accommodation had the greatest impact, it helped English language learners as well as non-English language learners, thus not narrowing the achievement gap. When no extra time was provided with a glossary, English language learners’ scores were actually lower, possibly as a result of information overload. The most significant finding is that linguistic modification was the only accommodation that narrowed the gap between English language learner and non-English language learner scores. Similarly, Rivera and Stansfield’s (2004) study of elementary science assessments in Delaware found that there were no statistically significant differences in scores on regular and simplified tests for non-English language learners, thus supporting the theory that linguistic simplification does not affect the comparability of scores.
Other studies, however, show us that linguistic simplification can still not be considered the solution to the issue of fairly measuring English language learners’ content knowledge. After finding that the lower performance of English language learners on NAEP math items was in large part because of the linguistic complexity of test items, Abedi et al. (1997) administered a simplified version and found that whereas students in the lower and intermediate-level math classes benefited from linguistic simplification, those in the highest level did not. This finding brings into question several other issues about the relationship between students’ language ability and math level. Do the students in the higher level math class also have a higher English proficiency level? Have they been in the United States longer? How much schooling, in their home language or in English, have the students had? It is becoming clearer and clearer that these and other issues are relevant to finding appropriate accommodations. If there is still a performance gap between English language learners and non-English language learners in the higher level math classes, how can it be reduced if linguistic simplification is not helpful?
In summary, the linguistic complexity of items on math and science assessments has been found to be a central factor in the overall lower performance by English language learners. Modifying, or simplifying, these items has generally proven not to affect construct validity, but only in some cases has it also been found to narrow the performance gap. Nevertheless, this accommodation is still one of, if not the, most promising in terms of leveling the playing field between English language learners and non-English language learners without giving either group an unfair advantage. In the cases where it is not helpful, we need to continue exploring other options.
Dictionaries
The research on dictionaries as a testing accommodation is multifaceted, looking at published dictionaries and customized ones, administered both with extra time and without. Dictionaries are tools that are potentially more useful for learners of intermediate proficiency levels than for beginners or advanced learners (Albus, Thurlow, Liu, & Beilinski, 2005) because low-proficiency students may not be able to access enough of the context to make the dictionary useful or may not have experience using dictionaries, and more advanced students may have the vocabulary but might need more processing time or simplified structure.
In a study by Abedi, Courtney, Mirocha, Leon, and Goldberg (2005), 611 fourth- and eighth-grade students took NAEP-style science tests with either no accommodation, an English dictionary, a bilingual dictionary, or linguistic modification. Findings indicated that non-English language learners were not affected by the accommodations, lending support to their validity. Furthermore, English language learners at both grade levels did perform better under an accommodation. Of all the accommodations implemented in this study, the English dictionary was the most effective for fourth-grade students but less so for eighth graders, for whom linguistic simplification was more effective. Whereas fourth-grade English language learners receiving the bilingual dictionary accommodation performed better than did those without any accommodation, they did not perform as well as did those with the English dictionary. In contrast, eighth graders with the linguistically modified test version scored highest, those with English dictionaries second best, those with no accommodation third, and those with the bilingual dictionary lowest. It therefore seems that many factors, including grade level and proficiency level, determine which accommodation works best.
Similarly, in Albus et al.’s (2005) investigation of the influence of dictionary usage on reading test performance for Hmong English language learners, findings indicated that dictionaries helped English language learners with an intermediate level of English reading proficiency and some dictionary skills, but students with lower and higher language proficiency did not benefit as much (Albus et al., 2005). In addition, students who spent more time with the accommodation did better than did those who used the dictionary but spent less time. In this particular study, because Hmong students’ native language literacy is low, a monolingual English dictionary was appropriate. For other language groups, a bilingual dictionary may be more suitable, but this brings with it some limitations in feasibility, for it soon becomes costly to provide dictionaries for every language group. More research is needed to focus on how students of different proficiency levels and with different background characteristics respond to the various types of accommodations.
Student Background Characteristics and Perceptions of Accommodated Tests
Not only is it important to establish the usefulness of the accommodations themselves, but it is equally essential to understand how individual student characteristics could impact the effectiveness of these accommodations. In a recent study, Kopriva, Emick, Porfirio Hipolito-Delgado, and Cameron (2007) investigated whether individualized accommodation assignments to English language learners improved their scores on a math assessment. The accommodations in this study included different combinations of a picture dictionary, oral reading of test items in English, and a bilingual glossary. The individual characteristics considered included language proficiency (English reading and listening, L1 reading), cultural proximity, and U.S. schooling factors. Kopriva then compared the performance of students who received recommended accommodations given their background with those who received accommodations not recommended given their background, and those who did not receive any accommodations. Findings indicated that students who received the recommended accommodations package performed best. Kopriva et al. also found that “inappropriate assignment of accommodations across ELLs [English language learners], without giving consideration to their ELP-R [English language reading proficiency] or L1-R [first language reading ability], appears to be no more useful than receiving no accommodations at all” (p. 18). It is critical to take these results seriously and to put energy into identifying which accommodations are best for different subgroups of English language learners.
How can we determine if the students themselves feel that the chosen accommodations are actually helpful? Relatively few of the studies on accommodations have used student feedback, but for those that have, valuable additional information about the effect of accommodations has been revealed. Abedi et al. (1997) did a Student Perceptions Study, conducting interviews with 38 eighth graders (native and nonnative English speakers) by asking participants to express a preference during structured interviews for either original test items or linguistically simplified items. A significant number of participants expressed that the simplified items were easier to understand than the original, more complex items. Similarly, Albus et al. (2005) used a posttest questionnaire and learned that although most students did not take full advantage of the provision of a dictionary accommodation, most felt that access to it would be beneficial. Knowing how students view the accommodations can prove extremely useful in determining the way in which the accommodations contribute to the face validity of the modified testing conditions, the overall testing experience, and the accommodations’ general perceived usefulness.
Conclusion
Even in light of the growing body of research on the effect of different accommodations on English language learners’ large-scale content assessment performance, there seems to be a great lack of systematicity in the choice of accommodations for individual learners. Therefore, not only should accommodations be selected based on their validity for the English language learner population as a whole and their effectiveness in narrowing the achievement gap, but they also should account for the differential effect of certain accommodations on different subgroups of English language learners. Future research on accommodations should specifically focus on differential effects across English language learners of different language proficiency levels, and it should make more use of qualitative studies to investigate how test-takers process the accommodations.
References
Abedi, J., Courtney, M., Mirocha, J., Leon, S., & Goldberg, J. (2005). Language accommodations for English Language Learners in large-scale assessments: Bilingual dictionaries and linguistic modification (CSE Tech. Rep. No. 666). Los Angeles: University of California, Center for the Study of Evaluation.
Abedi, J., & Gándara, P. (2006). Performance of English Language Learners as a subgroup in large-scale assessment: Interaction of research and policy. Educational Measurement: Issues and Practice, 25(4), 36-46.
Abedi, J., Hofstetter, C. H., & Lord, C. (2004). Assessment accommodations for English language learners : Implications for policy-based empirical research. Review of Educational Research, 74(1), 1-28.
Abedi, J., Lord, C., Hofstetter, C., & Baker, E. (2000). Impact of accommodation strategies on English language learners’ test performance. Educational Measurement: Issues and Practice, 19(3), 16-26.
Abedi, J., Lord, C., & Plummer, J. (1997). Language background as a variable in NAEP mathematics performance (CSE Tech. Rep. No. 429). Los Angeles: University of California, Center for the Study of Evaluation. Retrieved December 1, 2007, from http://www.cse.ucla.edu/products/Reports/TECH429.pdf
Albus, D., Thurlow, M., Liu, K., & Bielinski, J. (2005). Reading test performance of English-language learners using an English dictionary. The Journal of Educational Research, 98(4), 245-254.
Butler, F. A., & Stevens, R. (1997). Accommodation strategies for English Language Learners on large-scale assessments: Student characteristics and other considerations (CSE Tech. Rep. No. 448). Los Angeles: University of California, Center for the Study of Evaluation.
Kopriva, R., Emick, J., Porfirio Hipolito-Delgado, D., & Cameron, C. (2007). Do proper accommodation assignments make a difference? Examining the impact of improved decision making on scores for English language learners. Educational Measurement: Issues and Practice, 26(3), 11–20.
Rivera, C., & Stansfield, C. (2004). The effect of linguistic simplification of science test items on score comparability. Educational Assessment, 9(3&4), 79-105.
