iMoat: MIT Online Assessment Tool

Dates

September 2001 — December 2003

Principal Investigator

Dr. Leslie Perelman (Program in Writing and Humanistic Studies)

Problem

College writing placement exams are generally held in large classrooms without computer resources, failing to replicate the writing situations students will encounter in college.

Goal

iMoat's goal is to enable academic institutions throughout the world to administer and evaluate more valid online tests of writing and other academic skills.

Overview

iMoat is a collaborative effort with other colleges and universities to create a nationally accessible web service, built on .NET technology, to facilitate the assessment of the actual set of writing skills that students will need both in college and in their professional careers. This service allows tests to include readings similar to those which students encounter in college courses. Students take the test at home using a computer and familiar word processing software with enough time to think about the topics, plan their essays, and revise them. Evaluators design tests that reflect their own institutions' educational philosophy. Administrators reduce the hardware, software, and administrative costs of providing online testing. Colleges, as a result, have the opportunity to share test experiences, materials, and best practices.

MIT has formed a consortium with four other universities (CalTech, University of Cincinnati, Louisiana State University, and DePaul University) which, in the summer of 2002, tested a beta version of this national service. This group has been expanding to an ever increasing number of universities and colleges since then, and now is currently composed of seven schools: CalTech, University of Cincinnati, Louisiana State University, SUNY Stonybrook, Loyola University of Chicago, Albany College of Pharmacy, and Rice University.

Although this initial effort was focused on writing tests, this service is capable of expanding to cover any sort of evaluation situation in which students need time and resources unavailable in a standard classroom during a timed examination. Although these tests may never completely replace timed examinations, they have become a complementary form of assessment in writing, with future possibilities not only for fields such as history, literature, anthropology and political science, but also for science and engineering as well.

One central objective has been to create a highly flexible system that will easily adapt to the institutional needs and educational practices of significantly different colleges and universities. Although there is a central service, each institution is able to define and design its own assessments within its own distributed virtual space. Another objective of the project has been to provide a platform for colleges and universities to share best practices in writing assessment. Finally, over time the service has created an increasingly extensive digital database of student essays that is quickly becoming a significant resource for research in writing.

Conclusion by the Principal Investigator

We have demonstrated that:

• A national online essay evaluation service is practical and economically viable;
• The service is adaptable to address the specific needs of substantially different academic institutions, ranging from multi-tiered open-admission public universities to elite schools;
• Faculty at all institutions that have already used the service report that it is a substantially better and more valid form of writing assessment than the traditional pen-and-paper impromptu test;
• There is considerable interest from a number of academic institutions in using the service for placement of entering students.

"Other universities are following MIT's lead in using the Web to assess student writing abilities. This innovation, already known among writing program administrators as "the MIT Model" allows colleges to design tests for their institution that reflect each institution's educational philosophy and closely resemble the kinds of writing situations its students will encounter during their educational and professional careers. "—Dr. Leslie Perelman

Project Output

Publications

"Assessment in Cyberspace," McGraw-Hill Teaching Composition, 11/01/2005. View

Press

"Academia gets creative with Web services," CNET News.com, 10/27/03. View

Presentations

Panel at the Annual Meeting of the Conference on College Composition and and Communication. Chicago, March 22, 2002. "Connecting the Test to the Street: Online Evaluations as a Valid and Class Neutral Form of Assessment." Audience: 35 University Writing Program Administrators. View

Links

iMOAT: MIT Online Assessment Tool