AfriEcon

Project Details

We are building a Benchmark Dataset for Africa Economic Analysis based on official World Bank country reports. Your assessments ensure the reliability of this research resource.

Expert Annotator

0 Reviewed

-- Items Left

Annotator Guidelines

Self-Contained: The question must mention the country and year to be marked as "Excellent".
Grounding: Any numbers in the answer must exist exactly in the evidence.
No References: Avoid phrases like "Based on the text" or "According to the report".

Current Question ITEM-ID

Loading research question...

Proposed Answer

Fetching gold standard answer...

Factual Evidence Source Doc

Analyzing source metadata...

Annotation Form

Please review the details carefully before submitting.

1. Is it Factual?

✓

Grounded Supported by the text

✗

Hallucination Not found in evidence

2. Question Quality (1–5)

1 Discard 2 Poor 3 Average 4 Good 5 Excellent

3. Observations & Notes

Quality Scoring Scales

Excellent / Professional Mentions country & year explicitly. Fully self-contained and ready for automated benchmarking.

High Quality Clear, accurate, and professional. May have extremely minor phrasing simplicity.

Average / Acceptable Correct but uses vague references like "the document" or "this country" without naming them.

Poor / Unprofessional Awkward phrasing, poor grammar, or difficult for a human reviewer to understand quickly.

Serious Error / Discard Nonsense text, code snippets, or completely broken logic. Must be discarded.

Fact-Checking Guidelines

Check that every number in the answer appears in the evidence.
Ensure the answer doesn't add external information not found in the text.
Mark as Hallucination if any part of the answer contradicts the source.

💡 The Self-Contained Test

If you read the question to someone who hasn't seen the report, would they know exactly which country and year it refers to? If not, the score must be below 5.

General Tips

Avoid questions that start with "Based on the text..." or "According to the report..." — these should be scored 3 or lower.
Keep observations objective in the notes section.

Loading research question...

Annotation Form

Quality Scoring Scales

Fact-Checking Guidelines

General Tips

🎉 Evaluation Complete!