TempEval-2

Evaluating Events, Time Expressions, and Temporal Relations

This is the home page for Tempeval-2, task #13 of the Semeval-2010 competition. Tempeval-2 is a follow-up on Tempeval-1, which was an initial evaluation exercise based on three limited temporal relation tasks.

Summary

Newspaper texts, narratives and other texts describe events occurring in time, explicitly and implicitly specifying the temporal location and order of these events. Text comprehension requires the capability to identify the events described in a text and to locate them in time.

We provide three tasks that are relevant to understanding the temporal structure of a text: (i) identification of events, (ii) identification of time expressions and (iii) identification of temporal relations. The temporal relations task is further structured into four sub tasks, requiring systems to recognize which of a fixed set of temporal relations holds between (a) events and time expressions within the same sentence (b) events and the document creation time (c) main events in consecutive sentences, and (d) two events where one syntactically dominates the other.

The annotation scheme used is based on TimeML. TimeML (http://www.timeml.org) has been developed over the last decade as a general multilingual markup language for temporal information in texts and has been accepted as an ISO standard.

Data sets are provided for six languages: English, Italian, Spanish, Chinese, Korean and French. The data sets do not comprise a parallel corpus. Sizes range from 10K to 60K tokens. Participants can choose any combination of the three main tasks and the six languages.

Please read the task proposal (pdf file) for some more background and a slightly more detailed overview of the tasks.

Data

The training data release notes and the test data release notes are available on line. The data themselves are posted on the Semeval-2010 website. A final combined release is available on the timeml.org website in the corpora section.

Tempeval Google Group

Updates, bug reports on included scripts, and other messages will be posted here and at the TempEval Google group. Enter your e-mail here to join the TempEval group and mailing list: