More Tyranny of Metrics
Another excerpt from my forthcoming review:
Another form of metric fixation that appears in higher education is the desire to have some sort of "measurement" for all aspects of student learning. Courses should set out a variety of goals, and have a numeric score for how far the course went in meeting the goal. These "measurements" are then compiled, averaged, their spread measured, and so on. The joke here is that the numbers used as input are not measurements at all: professors are simply asked to pick a number between one and five, or one and ten, as to how close the class came to achieving the goal. Rather than being an actual measurement, the "score" is simply whatever number the professor being asked wishes to pick! It as though quantum physics were done by asking a bunch of physicists "How highly would you rate the attraction of leptons?" and then running calculations based on the results. In fact, what usually happens is that everyone asked to "score" these categories on a scale of 1-5 puts down 3 or 4 for almost everything: 1 or 2 would mean you are doing a bad job, and 5 would look like you are exaggerating! Professors simply write down the numbers that they think the administrators doing the "measuring" want to see. And the administrators then report these "measurements" to the university board, as evidence that "We are not doing too badly, but we've still a ways to go."
As Muller puts it, such efforts "combine hard measures of statistical validity with weak interest in the validity of the units of measurement" (72).
Another form of metric fixation that appears in higher education is the desire to have some sort of "measurement" for all aspects of student learning. Courses should set out a variety of goals, and have a numeric score for how far the course went in meeting the goal. These "measurements" are then compiled, averaged, their spread measured, and so on. The joke here is that the numbers used as input are not measurements at all: professors are simply asked to pick a number between one and five, or one and ten, as to how close the class came to achieving the goal. Rather than being an actual measurement, the "score" is simply whatever number the professor being asked wishes to pick! It as though quantum physics were done by asking a bunch of physicists "How highly would you rate the attraction of leptons?" and then running calculations based on the results. In fact, what usually happens is that everyone asked to "score" these categories on a scale of 1-5 puts down 3 or 4 for almost everything: 1 or 2 would mean you are doing a bad job, and 5 would look like you are exaggerating! Professors simply write down the numbers that they think the administrators doing the "measuring" want to see. And the administrators then report these "measurements" to the university board, as evidence that "We are not doing too badly, but we've still a ways to go."
As Muller puts it, such efforts "combine hard measures of statistical validity with weak interest in the validity of the units of measurement" (72).
Comments
Post a Comment