Measured
On the production service, 28 October 2026, against the held-out test split of the public iApp Wikipedia QA dataset: 739 questions on 192 Thai Wikipedia passages, each with one human reference answer. Token F1 is the overlap of words between the answer and the reference, from 0 to 100%.
| Measure | Result |
|---|
| Token F1, 739 questions | 82.4% |
| Exact match | 53.2% |
| Token F1, 500 XQuAD Thai questions (professionally translated) | 81.2% |
| Lowest question types: how many / how much, where | F1 77.8%, 76.7% |
| Declines when the passage does not hold the answer, 200 questions | 90.5% |
| Answer not supported by the passage, same 200 questions | 7.0% |
| Median time through the API | 1.70 s |
Exact match is lower mainly because the answer and the reference differ in extent, such as an added unit, not in content. Where an unsupported answer must never be shown, check the answer against the passage. Method, question types and errors: the benchmark (PDF).
Limits
- Thai text.
- When the passage does not hold the answer, the service says so in 90.5% of measured cases; in the rest it returns an unsupported answer or the question itself.
Data handling
Consent and notice under PDPA and GDPR remain with you as the data controller.