
	MWE 2008 Shared Task -- Evaluation Package

This package contains an evaluation utility function for the R statistical
environment [http://www.r-project.org/], which can be used to draw
precision-recall graphs, save them in PDF format, and calculate average
precision.

In order to use the script, participants must provide files in TAB-delimited
format with a single header row (precise column names do not matter), with 
the following information for each candidate in a data set

 - manual annotation from gold standard
 - score that determines ranking of candidates (high scores = good cand.)

Multiple scores may be specified if different algorithms are evaluated
(e.g. rankings according to multiple association measures).

Files in the evaluation package:

   precision_recall.R    R function for precision-recall graph
   example.R             sample R script explaining usage & eval procedure
   de_an.rankings.tbl    sample data file with automatic candidate rankings

To get started, open the R statistical environment, set the working directory
to the current directory, and then step through the sample script "example.R".

If you have any questions, please contact me at: stefan.evert@uos.de

Stefan Evert,
12 March 2008.
