Citizenship Test Revision Will Create New Barriers to Naturalization
(A new update to this article is available here.)
USCIS (U.S. Citizenship and Immigration Services) announced on December 15, 2022, a nationwide trial of planned changes to the naturalization test, including a civics exam with updated content and format, and a new English-speaking test. Although part of a stated and well-intentioned effort to improve the naturalization process in keeping with the Executive Order on Restoring Faith in Our Legal Immigration Systems and Strengthening Integration and Inclusion Efforts for New Americans, I believe these changes introduce two tests that will have the opposite effect and create new obstacles in the path to citizenship.
For the civics exam, the current straightforward set of oral civics questions and answers would be replaced by a reading-intensive multiple-choice format requiring a significantly higher level of language proficiency and test-taking skill. For the English test, the current evaluation of speaking ability that occurs during the normal course of the interview would be replaced by a new speaking test that elicits English about everyday topics and situations not related to civics, uses color photographs as prompts, requires separate test preparation, and potentially contains a scoring rubric more appropriate for placement and assessment of students in general life skill adult English classes rather than providing an evaluation of English ability pertinent to the context of naturalization.
A New Reading-Based Civics Exam
A multiple-choice civics exam would require reading comprehension skill at a significantly higher level of English proficiency and reading vocabulary knowledge than is currently required for naturalization. The current reading test is an oral test that evaluates an applicant’s ability to read aloud short sentences composed of a limited number of words, which the applicant is able to recognize and sound out as familiar sight words (from a short USCIS-provided list), or which the applicant is able to decode and sound out by applying knowledge of letter-sound relationships. Using the U.S. Department of Education National Reporting System (NRS) Educational Functioning Level Descriptors as a guide, the level of English proficiency required for such basic reading skill and knowledge of simple and familiar vocabulary is ESL Level 2 (Low Beginning ESL).* (In the linked NRS Technical Assistance Guide, see Exhibit 2.2 Functioning Level Table — ESL Levels 1–6.)
Changing the current oral civics test to a multiple-choice format would result in a significantly more challenging exam in which reading skill would be a key determinant of the applicant’s ability to demonstrate the required civics knowledge. Using the NRS descriptors as a guide, the level of English proficiency would be ESL Level 4 (Low Intermediate ESL) — a two-level jump above the current oral reading test. (A multiple-choice civics exam would surpass ESL Level 3 — High Beginning ESL — because reading skills required at that level only involve sight words, common words, familiar phrases, and simple sentences.)
Here are three resources to evaluate the reading vocabulary requirements of a contemplated reading-based multiple-choice civics exam:
1. The current test’s Reading Vocabulary study sheet from USCIS contains a total of 64 words and expressions that applicants need to be able to read aloud in sentences. These include the following categories and numbers of words/expressions: People (2), Civics terms (14), Places (3), Verbs (12), and other Content words (11). In contrast, a potential Reading Vocabulary study sheet for the existing oral reading test and a multiple-choice civics test based on the current 100 official civics questions and answers would contain a total of up to 511 words and expressions. These include: People (43), Civics terms (202), Places (54), Verbs (83), and Other content words (81). A hypothetical Reading Vocabulary list for a revised naturalization test is available here. Even if some current civics vocabulary categories were reduced, such as the names of Native American tribes or Cabinet departments, the list of words would likely be seven or eight times the length of the current test’s study sheet.
2. The New General Service List (NGSL) consists of the 2801 most important high-frequency words in English. It is based on a 273 million word subsection of the Cambridge English Corpus. A total of 318 words in the current civics exam questions and answers appear on the NGSL list. Other words in the civics exam (more than half) are off-list, indicating the high level of vocabulary knowledge required to deal with these terms in a multiple-choice exam format. A listing of Civics Exam Vocabulary on the New General Service List is available here.
3. For reference, a complete alphabetical listing of all Current Civics Exam Vocabulary is available here.
A Multiple-Choice Civics Exam would be ___.
A. Too long
B. Too difficult
C. Too complicated
D. All of the above
An additional factor adding to the difficulty of the contemplated civics exam is the multiple-choice format, which involves reading through a stem (question) and multiple answer alternatives, including a correct answer and distractors (incorrect choices). This format arguably evaluates the applicant’s test-taking skills as much as it evaluates civics knowledge. These skills can include predicting an answer from the question stem before reading and evaluating the answer alternatives, eliminating distractors that are known to be incorrect, and then selecting (or if necessary, guessing) an answer. Preparing for a civics exam in this contrived format would involve taking practice tests that by necessity include exposure to civics facts that are correct and, by multiples, civics distractors that are wrong.
In contrast, the current oral civics exam evaluates the applicant in a simple and straightforward manner. The applicant’s civics knowledge literally speaks for itself as questions are posed and short answers are given orally. Candidates for citizenship prepare to answer these questions directly in ways that allow for self-study with flash cards, an app, or a simple list of questions and answers in separate columns. They can also practice with a family member, a neighbor, a co-worker, or other helper. (“Ask me the questions, and I’ll give you the answers.”) And since they provide the answers orally, there is an additional factor of pride in accomplishment as applicants who are English learners use their emerging spoken language skills to demonstrate their civics knowledge. This is lost in a reading-based multiple-choice test.
Furthermore, the amount of civics content that applicants need to know to succeed on a multiple-choice test would exceed the current civics knowledge requirement. The current test includes 100 questions and answers. Many questions have multiple acceptable answers, and applicants usually select, memorize, and rehearse one particular answer they will use. In a multiple-choice format, applicants would need to know all acceptable answers in order to select the correct one that happens to appear in a test item, as well as to determine which distractors to rule out. For example, consider this difference in level of difficulty:
Current test question 72:
Name one war fought by the United States in the 1900s.
(There are four acceptable answers. The applicant has practiced one of them: “Gulf War”)
Contemplated multiple-choice version:
Name one war fought by the United States in the 1900s.
A. Civil War
B. Mexican-American War
C. Korean War
D. Spanish-American War
(The applicant must know all five wars in the 1900s that are acceptable answers in order to select the one that the test developer has included in this item. To add complication, the applicant must also recognize that the distractors are incorrect because they are answer options for a different civics question about wars in the 1800s.)
Understand the multiple-choice item writer’s task: To surround correct civics facts with a greater number of incorrect but plausible distractors in order to construct a test. Compared with the current simple and straightforward direct-question and short-answer civics exam, this alternative format is more complicated and difficult to prepare for, and applicant performance is as likely to be an indicator of test-taking skill as civics knowledge.
A Picture-Based English Speaking Test
The current evaluation of an applicant’s English skills occurs during the normal course of the interview as the officer asks for and verifies information pertinent to the application for citizenship. A contemplated photograph-based English speaking test would not be appropriate for the following reasons:
1) An English speaking test would add a testing requirement that does not currently exist, creating an additional task for the naturalization interview and a new exam the applicant needs to prepare for.
2) Unlike the current civics, reading, and writing tests, the content of a contemplated English speaking test in which applicants describe photographs based on topics and situations of everyday life would not align with the context of the naturalization process and would require applicants to study general vocabulary to answer questions not related to civics or the application for citizenship. Picture-prompt speaking tests depicting situations of ordinary life are commonly used to place students into general English language classes or evaluate their progress, but they do not elicit the level of meaningful performance of English skills that applicants demonstrate when communicating their most important personal information during the normal course of the interview.
3) A photograph-based speaking test on everyday topics will not elicit an applicant’s English proficiency in as relevant or accurate a manner as the current speaking evaluation. Just as the current oral reading test requires a proficiency level of ESL Level 2 (Low Beginning ESL), the current English speaking requirement can also be met at this level (but seem higher) when evaluation is based on English used during the normal course of the interview. The applicant at Level 2 can understand simple commands and questions related to personal information, and can use vocabulary to meet immediate needs (in this case, the interaction with the officer). The applicant can communicate meaningfully and sufficiently throughout the interview using phrases and incomplete sentences, while demonstrating little or no control of English grammar.
In contrast, a photograph-based speaking test that includes vocabulary on topics beyond the immediate need and context of the interview is evaluating proficiency at ESL Level 3 (High Beginning ESL). In addition, scoring rubrics for such speaking tests often include criteria that distinguish Level 2 versus Level 3 proficiency based on the extent to which the examinee demonstrates some control of grammar — an inappropriate measure for the current English requirement.
The contemplated new speaking test appears to be similar to the BEST Plus Test, which also employs photograph prompts to elicit spoken English on everyday topics. If USCIS is using BEST Plus as a model, it should focus not on that test’s photograph-based sections, but instead on the initial locator questions about personal information. The examinee’s responses to the locator questions determine which form of the photograph-based sections to administer to the examinee.
A USCIS English-speaking performance evaluation (not test) that emulates the locator questions section of the BEST Plus Test could serve to standardize key interactions during the interview as the officer asks about and verifies the applicant’s personal information relating to naturalization. Officers can be provided with a list of questions that elicit basic personal information — asking for or verifying the applicant’s name, address, date of birth, country of birth, country of citizenship or nationality, contact information, family members (parents, children, current spouse), marital status, current employment, and current residence. (This aligns with recommendations to USCIS by the Citizenship Test Working Group on September 17, 2021.)
Implementing such a procedure would retain and better standardize the English-speaking evaluation as a demonstration of English ability during the normal course of the interview and eliminate the addition of a new test for USCIS to administer and the applicant to prepare for. As part of this procedure, paraphrased versions of the questions should also be provided. A simple scoring rubric can be used to evaluate the applicant’s ability to communicate information meaningfully and sufficiently (without regard to control of grammar). And a training module should be developed for implementation and periodic retraining to assure consistency and reliability.
Recommendations:
It is unfortunate that USCIS has begun a stakeholder input process after already determining the alternative test formats to be piloted. A step back would be prudent — to examine the rationale for the changes, evaluate the expectations regarding applicants’ level of English proficiency, and consider testing alternatives — including retention or improvement of the existing civics and English evaluations.
My recommendations are as follows:
1. Retain the existing civics exam in its oral short-answer format. Refresh the exam by replacing up to 15 percent of the 100 test items with questions of similar content and comparable level of difficulty. e.g., Replace “Who was President during World War I? (Woodrow Wilson)” with “Who was President when terrorists attacked the United States on September 11, 2001? (George W. Bush)”
2. Preserve the current evaluation of the applicant’s ability to use English in the normal course of the interview, but standardize the questions and apply a valid and consistent scoring rubric for determining whether the applicant has demonstrated the required level of English speaking skill.
3. If the trial testing train has already left the station, test data should be reviewed carefully to determine if these initiatives “raise the bar” and create new barriers to naturalization. Trial testing should include a special focus on examinees who have proficiency at ESL Levels 2 and 3, respectively, in order to determine — through experimental and control groups — the extent to which students at these levels are able to pass the current oral civics exam but cannot pass the pilot multiple-choice version. If the pass rate declines, these initiatives are in conflict with the goals of Executive Order 14012 to improve the naturalization process. In that case, see Recommendations 1 and 2.
4. Community-based organizations and other institutions that participate in the trial testing should monitor the impact of the new test formats on their students and programs, particularly at ESL Levels 2 and 3, with attention to amount of instructional time required to incorporate USCIS-provided educational materials, the English proficiency level and duration of enrollment of students who volunteer for the program, and pass rates of participants in the experimental and control groups suggested above. The role of these organizations is critical in assuring that participation in the trial is not skewed toward students at higher-than-average levels of proficiency, previous education, family resources, or other factors.
Opportunities to Provide Input to USCIS
You can submit written comments to USCIS regarding the testing changes at natzredesign22@uscis.dhs.gov.
Updates about the test redesign including announcements of upcoming public engagements are available at the USCIS Naturalization Test Redesign Development 2022 web page.
Bill Bliss has worked in English language and civics education since 1974 as a teacher, trainer, curriculum developer, and consultant, and has provided technical assistance on citizenship training and testing since the 1980s legalization program. He is the author of two civics courses: Voices of Freedom, for learners preparing for the naturalization process; and Basic Civics for U.S. Citizenship, a review text for secondary school students in states that require a civics exam for high school graduation. Bill is a member of the Citizenship Test Working Group and the Naturalization Working Group. The views expressed in this article are solely those of the author.
bill.bliss@languageandcommunication.org
LinkedIn
