Effect of Items Position Change on Students’ Achievement in the Ethiopian University Entrance Examinations (EUEEs)

Abstract

To combat exam malpractice in crowded examination rooms, the Ethiopian University Entrance Examinations have been administered in four coded booklets of different reshuffling of item orders. However, research has revealed that systematic item position changes have significant effects in achievement scores. The main purpose of this study was to find out if random item order reshuffling would also have mean achievement score differences depending on which of the exam booklets test-takers were tested with. To address this purpose, the Entrance Examination 5 subjects (English, Mathematics, and 3-sciences) for 6-years for 21 sample public schools (11,376 grade 12 students) was received from National Education Assessment and Examinations Agency. In addition to the usual descriptive statistics, the data was analyzed with Spearman’s rank-order correlation to determine if the item distributions in the four booklets of the same exam significantly differ with each other. Besides, one-way ANOVA was used to determine if there are statistically significant differences in students’ achievement mean scores by booklets. The Spearman’s rank-order analysis shows weak to moderate item position order differences among booklets. In contrast to this, statistically significant mean achievement differences were found in 66.67% of the exams, which put at a serious disadvantage up to 16.64% of test-takers due to which exam booklets they were tested with. Hence, it was recommended that all stakeholders: test developers, exam booklet developers, result publishers and decision makers be aware of the unfairness of the current practice with item reordering and therefore take appropriate compensatory measures.