2019 Open Cross Lingual IR Prize Challenge
Sponsored by IARPA

OpenCLIR (Open Cross Lingual Information Retrieval) Prize Challenge seeks innovative approaches to locate information in audio and text documents in low-resource languages, using English queries. This capability is one of several expected to ultimately support effective triage and analysis of large volumes of data, in a variety of less studied languages. Successful systems will be able to adapt to new languages and new document modalities and genres.
The OpenCLIR evaluation was created out of the IARPA MATERIAL Program that encompasses more tasks, including domain classification and summarization, and more languages. The purpose of OpenCLIR is to provide a simplified, smaller scale evaluation open to all. Please see IARPA's MATERIAL website for more information on the MATERIAL program.
Prizes and Judging Criteria
The winning submissions will be determined using a combination of score above a provided baseline score (80% towards winning prize) and rating of system description (20% towards winning prize). Submissions by participants who fail to submit the required system description will not be eligible to win.
  • First Place Prize on speech documents – $20,000
  • First Place Prize on text documents – $10,000
Prizes will be awarded in accordance with the laws of the USA and of the winning participants’ countries. More detailed rules regarding the awards may be specified in a separate document.
Given a set of English queries and a set of non-English documents, determine documents that are relevant to the queries. The language ID of the documents is revealed when the development data is released.
How to Participate
To take part in the OpenCLIR Prize Challenge, you need to register on the openclir.nist.gov website and sign the data agreement. To sign in or sign up for an account, click "Sign In" or "Sign Up" top right. Once the agreement is approved, you will be sent information on how to download the development data and, at the appropriate time, the test data.
Please see NIST's OpenCLIR information website for more information on OpenCLIR19, including the evaluation plan, schedule information, and additional resources. If you have any question, please send an email to openclir_poc@nist.gov
Release of Evaluation Plan
Registration Begins
Release of ANALYSIS, DEV, & QUERY-DEV (encrypted data & decryption keys)
Server accepts ANALYSIS, DEV, & QUERY-DEV Submissions.
Registration Period Ends
Release of EVAL & QUERY-EVAL(encrypted data)
Release of decryption keys and server accepts EVAL & QUERY-EVAL Submissions
System output due to NIST
System descriptions due to NIST