Project two involves examination of 4 test items from a fifth grade math test. After conducting an item analysis of the four items, I came to the conclusion that Item #1 should be either heavily revised or discarded; Item #2 needs to have distractor B revised; and Item #3 also needs to have distractor B revised. Item #4 is a good test item, and could be used again with confidence. The paragraphs which follow include the details of my item analysis.
For Item #1, 40% of students in the upper 25% answered correctly, while 50% of students in the lower 25% answered correctly. Therefore, the item discrimination index is -0.10, using the calculation: D = 0.40-0.50 = -0.10, which indicates a problem with this item. The item difficulty index is 0.45, meaning that 45% of the students answered this item correctly, which is low for a multiple choice item with 4 choices. The distractor analysis shows that more students from the lower 25% selected A, but more students from the upper 25% selected B, and no students chose D. Since effective distractors should attract more examinees from the bottom group than the top, A is all right, but B needs to be changed because more students from the upper 25% chose B. Also D obviously is not doing its job as a distractor, because no students chose D. Based upon item discrimination index and distractor analysis, this is not a good item. It should be revised or discarded.
For Item #2, 75% of the upper 25% of students and 73% of the lower 25% of students answered correctly. The item discrimination index is .02; calculated as follows: D = 0.75-0.73 = 0.02, which is quite low. The item difficulty index is 0.74, which means that 74% of students answered this question correctly; 0.74 is an acceptable item difficulty index for a multiple choice item with 4 choices. A distractor analysis shows that the lower 25% of students selected C and D more frequently than the upper 25% of students; however, more students in the upper 25% than students in the lower 25% chose item B. Although this item has a low discrimination index, its item difficulty index is acceptable. Distractors C and D are doing their job, however, distractor B should be revised if this item is used again.
For Item #3, 52% of students in the upper 25% answered correctly and 16% of students in the lower 25% answered correctly. This means that the item discrimination index is 0.36 (D = 0.52-0.16 = 0.36), which is good. The item difficulty index is 0.45, meaning 45% of students answered this item correctly, which is low for a multiple choice item with 4 choices. A distractor analysis shows that more students in the lower 25% selected C or D than did students in the upper 25%. No one chose distractor B, however. Overall, this item has a good item discrimination index, a low item difficulty index, and 2 distractors are working as they should. Distractor B should be revised if this item is used again.
For Item #4, 93% of the students in the upper 25% answered this item correctly and 16% of students in the lower 25% answered this item correctly. The item discrimination index is 0.77 (D = 0.93-0.16 = 0.77) which is excellent. The item difficulty index is 0.50 which is low for a multiple choice item with 4 choices. A distractor analysis shows that more students from the lower 25% selected distractors A, B, and C than did students from the upper 25%, which means that these distractors are doing their job. This is a good item, based on item discrimination index, and distractor analysis. The only potential revision might be to distractor B, since no students from the upper 25% selected B.
Conducting an item analysis can be time-consuming; however, the process is worthwhile when the test will significantly impact student grades and the teacher wishes to feel confident in reporting the scores from the test. With the current trends toward common standards and mastery based learning, this type of certainty about how well grades indicate mastery of a skill or concept will become more and more important for educators.