|
A few weeks ago I wrote a column about “SCIgen,” a
computer program dreamed up by three Massachusetts Institute of
Technology whiz kids to generate phony research papers. The result
is pure gobbledygook, but one of SCIgen’s electronic brainstorms was
accepted for presentation at the World
Multi-Conference on Systemics, Cybernetics and Informatics this
summer.
The offer was withdrawn when
the hoax became public, which, I lamented, was a crying shame. The
students had probably made the greatest discovery of the
conference: unintelligible mumbo jumbo is not good science.
Now comes a story from Associated Press about the use of
software to score essays written by college, high school, and grade
school students. Ed Brent, a sociology professor at the University
of Missouri-Columbia uses a program he wrote called “SAGrader” to
score essays in his “Introduction to Sociology” course. SAGrader
tallies the key points Brent identifies, analyzes the supporting
arguments, and voilà!, out spits a score.
Students have used technology to shortcut essay
assignments for years, so perhaps turnabout is fair play. Papers on
every conceivable subject are available on the Internet for a fee or
on sites that borrow the it’s-not-stolen-if-you-don’t-catch-me
morality of file sharing sites used to steal copyrighted music and
movies. Teachers struck back with software that detects purloined
papers and plagiarism; it’s only logical that computers take over
grading completely.
Brent and his teaching assistants still assign final
grades the old fashioned way, and in any event SAGrader is not the
only game in town. A company called Educational Testing Service
sells a program called e-Rater that is used in U.S. school districts
with a total of 500,000 elementary, middle, and high school
students. The software is also used to score an essay on the GMAT,
the admissions exam used by most graduate-level business schools.
AP reports that when the University of California, Davis
tried e-Rater, a suspicious employee devised a test. He submitted a
student’s letter of recommendation as a paper on workplace injuries,
substituting the phrase “risk of personal injury” for the student’s
name. He scored five points out of a possible six.
Stunned, the man resubmitted the paper with the word
“chimpanzee” inserted at random. He got a perfect score.
I wouldn’t rule out the possibility that California has
a problem with injured chimps in the workplace but my bet is that
e-Rater still has a few bugs. I’ll be sure to mention it to the
next MBA I meet who brags about acing the GMAT.
I, on the other hand, would be delighted to submit one
of my columns to grading software; my work is already regularly
critiqued by readers claiming to be teachers or professors of
everything from English and biology to driver’s education. If
that’s you and you have access to an automated scoring program,
email me at
brent@brentmorrison.com and we’ll give it a whirl.
While we’re at it, I’d also like to crank through my
footnoted, complete-with-charts-and-graphs research paper titled
“Von Neumann Machines No Longer Considered Harmful.” I churned that
out on the website of the MIT jokers who wrote SCIgen and graciously
make it available to the public without charge. I didn’t mention
that little service in my column on the hoax, but I did file it away
in case I get really, really, desperate to meet a column deadline.
Just kidding. My editors actually read my work, not
that they don’t let me get away with my share of gibberish.
The AP story indicates grading software is generally
used for screening purposes and that real people determine final
scores. But the day is coming when students will turn in computer
generated essays that will be scored by their schools’ grading
software, completely untouched by human hands. When that day
arrives, we will have once again outsmarted ourselves.
|
