Tinkering with Rasch Parameters
Returning to the original Rasch definition of person and item parameters, given by:
| θvi | = | δi/ζv | (12) |
Rasch puts flesh on them by defining a test, test zero, for which δ is unity. In this case:
| θv0 | = | 1/ζv | (16) |
He then says: "the ability of the person is the reciprocal of the probability of misreading in the reference test". Let's not think too hard about the reference test for a minute, but say the probability is estimated at 5%. I've initially chosen a low value, because Rasch uses Poisson estimation, which relies on low values. That returns a value of 20 for ζ.
Rasch then looks for a person with unit ability. Rasch uses this conceptual person to estimate test/item difficulty as follows:
| θ0i | = | δi | (17) |
He then adds the verbal definition: "the difficulty of a test is the probability that the standard person makes a mistake in the test". Let's keep θ low again and say we have a test for which δ is 5%. Now lets combine that test with the person for whom 1/ζ is also 5%. We now have:
| θ | = | 0.05/20 |
The probability of an error in this case is 1/400 or 0.25%. You could force a similar result in my bean model, when there are 5 blue beans and 95 red beans in both person and test/item boxes, by selected a bean from each box in turn and stipulating that selecting a blue bean from both boxes constitutes an error. If on the other hand you mix the contents of both boxes and select a single bean from the combination, the probability of a blue ball (which represents an incorrect answer) is 5%, which is intuitively a bit less pleasing than the result from the Rasch method.
Let's try the same procedure where both θv0 and θ0i are 50%. In this case δ is 50% and ζ is 2. In the Rasch model the probability of an error is 25%, as it would be if I wanted to draw a blue bean from both an item box containing 50 blue beans and a person box containing 50 blue beans. But if I mix my boxes and select a single bean, the the probability of a blue ball (which represents an incorrect answer) is 50%. This I find intuitively more pleasing than the result from the Rasch method. Combining a median pupil with a median item/test, I would look for a median type outcome - i.e. a score around 50%; not 75%, as predicted by the Rasch method.
If we look at a very difficult test, where θ0i is 95%, and a very challenged pupil, where 1/ζ is also 95% (and ζ is 1.053), θ in the Rasch model is 90.25%. I could force a similar result in my bean model, when there are 95 blue beans and 5 red beans in both person and test/item boxes, by selecting a bean from each box in turn and looking for two blue beans. But if, as I prefer, I mix the contents of both boxes and select a single bean from the combination, the probability of a blue ball (which represents an incorrect answer) is 95%,
The result of the bean mixing method is the mirror image of the result of the same method applied to the very able pupil and the very easy test. The Rasch parameters do not produce the same symmetry. The able pupil and the easy test combined to give a probability of 0.25% for an error, while the challenged pupil and the difficult test combine to give a probability of 90.25% for an error. Intuitively you might expect the extremes of ability and easiness to combine to produce a very low probability for an error and the extremes of inability and difficulty to combine to produce a very high probability for an incorrect answer (and a correspondingly low probability for a correct answer).
Confusion perhaps arises because Rasch was talking about committing an error in a multi-word reading test, whereas much of the subsequent work (including my own) focuses on dichotomous results arising from addressing a single item. For simplicity of arithmetic, take a 20 word reading test. If the probability of committing an error on a single word is 1%, the probability of a perfect reading is approximately 82%, while probability of making a single error is approximately 16.5%. Note the lack of symmetry. On a single item, if the probability of success is 82%, the probability of error is 18%.
For a less able individual, if the probability of committing an error on a single word is 10%, the probability of a perfect reading is approximately 12%, while probability of making a single error is approximately 27%. Again, there numbers to not relate to each other as they do for success or failure on a single dichotomous item, but I am not sure how this helps us to understand the practical implications of the way Rasch defined his individual and item parameters. What it does is emphasise the importance of distinguishing between predicting results on dichotomous items and predicting results in multi item tests, reading or otherwise.
In his discussion, Rasch refers to the "probability of misreading" (op. cit. page 16) in a test, but he is a bit vague as to whether this a single error or several. Perhaps this will become clearer later in the text.
Comments