Guide
What Is FACET Translation Quality Evaluation?
What is FACET translation quality evaluation? FACET is a reference-free method from the WMT26 shared task. It scores a translation in three separate passes — fluency, accuracy and consistency. Each pass gets only the context its error type needs.
What is FACET in machine translation quality evaluation?
The FACET paper describes a system entered in the WMT26 Automated Translation Quality Evaluation Task. The task ran October 28–29, 2026 in Budapest, alongside EMNLP. FACET does not use one all-purpose judge model. Instead, it prompts a single fixed LLM three separate times: once for fluency, once for accuracy, once for consistency. Each prompt sees only the context that error type needs. The three sets of error spans are then merged. That merge produces the three required outputs: span annotations, segment-level quality scores, and error-free labels. No trained components are involved.
Why split fluency, accuracy and consistency into separate passes?
Different error types need different evidence. The FACET paper, published at WMT 2026, puts it plainly: whether meaning is preserved can only be judged against the source. Whether the target is well-formed, or names an entity consistently, can be judged from the target alone. A single prompt that checks everything at once forces a model to juggle those mismatched needs. FACET avoids that by running three focused passes instead. The authors also submitted FACET-C, a variant that drops the Consistency pass, to see how much that pass actually adds.
What each pass checks against
| Pass | What it catches | What it judges against |
|---|---|---|
| Fluency | Whether the target is well-formed on its own | Target text alone |
| Accuracy | Whether the meaning of the source was preserved | Source and target together |
| Consistency | Whether a term or entity is named the same way throughout | Target text alone, across the document |
This kind of reference-free, per-error scoring is also the layer FAS Localize (faslocalize.com) runs inside its own AI localization pipeline: AI drafts are scored by a QE gate, and native linguists verify the result before it syncs back to a client's repository, CMS or API.
What did FACET find when it was tested at WMT26?
FACET's system rankings placed post-edited human translation first. The Consistency pass changed about a tenth of segment scores. But it left the overall ranking nearly unchanged. FACET-C, the version without that pass, produced almost the same ordering. Fluency and accuracy alone seem to carry most of the ranking signal for most language pairs. The task itself continued the WMT 2025 move to unify the old Metrics and Quality Estimation tracks into three subtasks: error-span annotation, score prediction, and error-free segment detection.
Where does a QE score like this fit in a real translation pipeline?
Automated quality estimation is a routing signal, not a verdict. At FAS, a reference-free QE model scores every AI draft, and a native linguist post-edits it — light or full post-editing depending on the file, with high-stakes files getting a second-linguist review. FACET's three-pass design could make that scoring richer: a segment flagged only for consistency points a linguist straight to a terminology check, while one flagged for accuracy calls for a closer bilingual read. No reference-free metric catches client-specific requirements, though — regulatory phrasing, brand voice, or a glossary that changed last week. That is why a linguist stays in the loop on every project, even when the QE model says "error-free." See our AI localization pipeline for how FAS Localize blends automated scores with human review.
Related reads
- How error-free segment detection changes post-editing workflows
- LLM-as-judge vs. COMET: which QE signal trusts more?
FACET shows that a clever prompt split can squeeze more signal from one fixed model. It also reminds us that the last mile — client terminology, legal nuance, brand tone — still belongs to a human.
Frequently asked questions
Does FACET need reference translations to score output?
No. FACET is reference-free. It scores fluency, accuracy, and consistency using only the source and the translated text, with context tailored to each error type.
How many model calls does FACET make per segment?
Three. One for each error type. The same fixed model is prompted three times, each time with a different context window, then the error spans are merged.
What changed in WMT26 compared to earlier shared tasks?
The 2026 task unified the former Metrics and Quality Estimation tracks into three subtasks: error-span annotation, segment-level score prediction, and error-free segment detection.
Why does FAS Localize still keep a human linguist in the loop?
Automated scores like FACET are strong ranking signals. But they miss client-specific terminology, tone, and regulatory nuance that only a linguist can verify.
Sources
- FACET at WMT 2026 Automated Translation Quality Evaluation Task, arXiv (accepted at WMT 2026, EMNLP). Accessed Nov 12, 2026.
- Shared Task: Automated Translation Quality Evaluation Systems (WMT26), Eleventh Conference on Machine Translation (WMT26), statmt.org. Accessed Nov 12, 2026.
- FACET at WMT 2026 Automated Translation Quality Evaluation Task (ACL Anthology record), Association for Computational Linguistics, ACL Anthology. Accessed Nov 12, 2026.
Want localization to run on autopilot?
Talk to us about your pipeline
Dao Huy
Senior Project Manager · EN · ZH · FR to Vietnamese
Your point of contact from brief to delivery: he scopes the job, sets up the AI and native-review workflow, and keeps quality, deadlines and budget on track.