Moreover, it can bring several surprising benefits, such as improving the consistency of scoring and the possibility of providing instant feedback to students on their performance (Gierl, Latifi, Lai, Boulais, & De Champlain, 2014). When successfully implemented, AES can speed up the scoring process significantly. Then, the AES system attempts to learn a scoring pattern or a rule close to the human raters’ using those features. Such features often include the length of essays, number of words, word usage, and sentence complexity. To do so, the AES system has to identify deterministic linguistic features that human raters used to identify essay quality. Thus, the machine-replaced third marker can be trained based on how the scoring and grading was made by the other two human markers. Traditional scoring procedures often consist of a minimum of three scorers to ensure scoring reliability and the fundamental idea of AES was to introduce a system to replace the third marker thereby saving time and money.
Therefore, to promote the use of written-response tasks that can be used to evaluate student understanding in a creative and less restrictive way, overcoming such disadvantages stemming from scoring and administration procedures is critical in the digitally-based assessment.Īutomated essay scoring (AES) was first developed to help overcome this scoring and administration problems by encouraging cost- and time-efficient marking procedures of written-response questions (Page, 1967). They provide evidence of students’ composition and organization skills, grammatical knowledge, background knowledge, and analytic thinking and reasoning skills. However, written-response items do have many benefits. Compared to essays and other written-response tasks, which are prone to subjective scoring and which require more time for recording answers, selected-response questions can be scored more accurately and they require students to spend less time recording answers. With traditional paper-based assessment, selected-response items (e.g., multiple-choice questions) are often used because they are efficient to administer, they are easy to score objectively, and they can be used to sample a wide range of content domains in a relatively short time using a single test administration (Haladyna & Rodriguez, 2013 Rodriguez, 2016). Such items often require students to express their understanding in a creative way using their own words, thereby, invoking higher-order reasoning and complex thinking skills (Scully, 2017). The purpose of these new item types is to provide more authentic assessment opportunities for students (NAEP, 2018). For example, the National Assessment of Educational Progress (NAEP) introduced innovative item types−such as interactive scenario-based questions−with their new digital-based assessment environment. While the initial transitions have focused primarily on administrative benefits, such as increased test security and flexible testing schedules, the more recent transitions are focused on the use of new item formats. High-stakes testing is in the process of transitioning from paper to computer-based assessment.
0 kommentar(er)
