The Versant Test Development Team has Completed Custom Projects with Large Customers
Netherlands Ministry of Justice
The Need: The Dutch Parliament passed a policy in November of 2004 requiring that all persons seeking to immigrate to The Netherlands be able to demonstrate proficiency with spoken Dutch.
The Solution: The Versant Test Development team was selected by the Ministry of Justice's official partner, CINOP, to develop the Toets Gesproken Nederlands (TGN). The TGN is a fully-automated listening and speaking test designed to measure the ability of a test taker to understand basic Dutch spoken at a normal conversational speed, and to respond correctly and intelligibly in Dutch. This TGN is currently being delivered at Dutch embassies and consulates around the world. Read more...
U.S. Department of Defense (SOFLO, Spanish)
The Need: The U.S. Special Operations Forces Language Office (SOFLO), the exclusive agent for the U.S. Special Operations Command (USSOCOM) foreign language training program needed an efficient and accurate assessment of spoken Spanish proficiency.
The Solution: The Versant Test Development team developed a new Spanish language test similar to Versant's spoken English tests - telephone based with fully automated scoring. The Versant Spanish Test was developed with data collected from more than 500 adult non-native Spanish learners, including military personnel and university students, and more than 500 native speakers of Spanish. Subsets of these speakers were also assessed with a variety of other rating instruments. Results show that the Versant Spanish Test elicited sufficient spoken language behavior on which to base a reliable and accurate human judgment of practical speaking and listening skills. Furthermore, automatic scoring of responses from a Versant Spanish Test administration can produce reliable and useful information about these spoken language skills. This exercise demonstrated the Versant Test Development team's ability to leverage its core technology into other languages. The Versant Spanish Test is provided by Pearson and is also available commercially to other customers.
Harvard University (NCSALL) and ETS
The Need: The National Center for the Study of Adult Learning and Literacy (NCSALL) at Harvard University was planning a multiple-part test of reading-related skills in cooperation with ETS. The test, RTOLD (RAN, TOWRE, Oral Language, Digit Span), was designed to analyze skills like word and letter decoding, memory, and English speaking and listening skill, which are cognitive components of reading ability.
The Solution: The Versant system delivered all the listening, speaking, memory, and decoding tasks in the RTOLD test over the telephone and performed a range of other functions, from response collection to completely automated analysis of oral English proficiency. Approximately 1,000 people took the RTOLD test in major cities across the United States, sampling readers from households and from adult education programs. The speaking components of the RTOLD test were scored automatically by the Versant system. The word decoding and memory sections of the RTOLD comprised standard performance tasks, which were built into the Versant testing system and automatically delivered by phone to subjects. Scoring of the letter and word decoding and memory responses were conducted at Harvard and ETS. This deployment demonstrated the Versant Test Development team's ability to provide a solution that combines its delivery system with the flexibility of both automatic and human scoring.
U.S Department of Education (NAEP)
The Need: The U.S. Department of Education conducts a national study every year to analyze how well the nation's youth is progressing in basic academic areas (also known as "The Nation's Report Card"). The last major oral reading study, done in 1992 and administered by ETS/Westat, was delivered via tape recorders. The U.S. Department of Education needed a simple, dependable method to replicate that test and collect the student responses in 500 schools.
The Solution: The Versant Test Development team specified, designed, implemented, and tested a software system for NAEP which ran on notebook computers and was used to administer the tests. Students heard instructions over the computer and responded into a microphone. A test administrator navigated the software and handed reading and other materials to the student as required. The Versant team provided technical support to ETS during operation of the testing and during the transfer of data from the notebook computers. The Versant Test Development team set up listening/rating protocols through an Internet-based interface for scoring. The Versant team organized the response material into an appropriate hierarchical file structure and delivered it to ETS. The Versant Test Development team later produced additional automatic measures for the passage readings collected during the study. The measures represented three response features, Narrow Time, Initial WPM (Word Per Minute), and Run Rate. This project demonstrated the Versant team's ability to develop, deliver and administer a large-scale customized assessment that utilized computer delivery and its human rating management system.
U.S. Department of Education (NAAL)
The Need: The National Center for Education Statistics of the U.S. Department of Education intended to conduct a reading fluency supplement to the NAAL. The goal was to study basic reading in 18,000 adults across the nation to illuminate the processes that enable functional reading performances like passage comprehension and in particular to identify the skills that limit performance in the lowest level adult readers.
The Solution: The Versant team played two roles in the NAAL fluency study: (i) basic test development, and production delivery and scoring, and (ii) advanced data analysis, scaling studies and reporting. First, the Versant Test Development team implemented a series of tasks to be administered via laptop. These required participants to read characters, lists of words and pseudo-words, and passages. The recorded responses to these tasks were transferred to the Versant database, then analyzed and scored automatically by Versant technology. The standard scoring provides information about the participant's reading rate, articulation rate, and fluency-related measures including incidence of short and long pauses, word deletions and false starts. The research elements include a finer automatic analysis of word reading accuracy, with derivation of new scale scores, and public reporting of analyses of the observed performance data. This assessment program demonstrated the Versant Test Development team's ability to develop, deliver and administer a large-scale customized assessment that utilized computer delivery, as well as automatic scoring and custom data analysis tools.