It comes as no surprise to anyone who has ever been a teacher or a student to hear that students don't always have the time or will to complete their reading assignments. However, it seems clear that carefully read papers that make a big impact on students are the ones that go on to become most important in the field as those students become full-fledged scholars.
We would like to hypothesize about the anthropological aspects of the sociodynamics of this process in more detail, but the results of the current study indicate that such long-windedness will lessen the long-term impact of this paper. (See below!)
Seventy-five grad students in linguistics, from several universities, were each assigned to read 25 papers over the course of the academic year, then rank them based on the quality of their content. The papers where chosen so as to divide evenly into three categories based on their length: Short, Medium, and Long.
The aggregate results of the rankings are displayed in the graph. In general, the Long papers are tightly clustered around the middling scores. The Medium papers show more variability in their scores, and score better in general. Short papers continue this trend, showing even more variability, but again with a higher mean.
Collectively, this trend seems unlikely. The truth of the matter is that almost no one read the longer papers. Instead, they skimmed them or read the abstract, then gave them a conservative ranking (in the 5-6 range), modified by how well they thought they might like them (+/- 1 or 2 points) if they ever got around to actually reading them.
This analysis is backed up by the follow-up discussions for the reading assignments. Comments on Long papers are more likely to be vague and noncommittal: "interesting", "ok", "useful", and the almost daring "nothing earth-shattering". By comparison, comments on the Short papers are much more specific and detailed, even passionate:
This analysis is further bolstered by some interesting evidence: we also asked some of the students to predict their rankings based on the title, author, subject matter, and abstracts of the papers they were about to read. Long and Medium papers showed little variability between the predicted ranking and the ranking assigned after "reading" the paper. Short papers showed more variability, and surprise at the difference between expectations and final ranking much more often figured in the students' comments on the Short papers during discussion.
We could explore this additional evidence in more detail, but it would unfortunately cause this paper to exceed the vonn Güügënschnëchtën-d'Qi Optimal Article Length™.
Astute readers will note an odd upswing in the tail end of the otherwise felicitous curves fitting the data. This odd outlier data was found, upon closer inspection, to be from a single student. We were so intrigued that we tried to interview the student, but were unable. None of the faculty or the other students at this student's University could readily recall his name--he is simply known as "The Tool", even by the faculty. He spends all of his time in a study carousel at the library, has never attended a Grad House party, and never speaks to any of the other students on the rare occasions he sees them. He liked all the papers very, very much.
In our next paper we will consider the effects of graphics, both color and black and white, on paper rankings. In the meantime, interested scholars can send a check or money order for $269.95 to the authors to discover the detailed secrets of the vonn Güügënschnëchtën-d'Qi Optimal Article Length™.