Benchmarks

Each model's headline score on each test set, set against the strongest other system we measured in the same test, whether we beat it or not. The numbers are copied from the papers, where you'll find every baseline, the test conditions and what the scores don't show.

Netra OCR 1

Full resultsLimitations

  • KHOB

    325 real lines, the public Khmer OCR benchmark

    CER %, lower is better

    Netra OCR 1
    0.96
    Tesseract
    5.12

    Ahead

  • Legal Document

    220 lines from phone photos of certificates and ID cards, annotated by us

    Every other system gets more than 40% of characters wrong

    CER %, lower is better

    Netra OCR 1
    5.31
    Tesseract
    41.78

    Ahead

  • Printed Words

    500 short synthetic words in 10 fonts

    CER %, lower is better

    Netra OCR 1
    2.95
    Tesseract
    20.14

    Ahead

Netra MT 1

Full resultsLimitations

  • ALT, English→Khmer

    1,018 professionally translated sentences

    Ahead of every open model, by 7 points over the next best

    spBLEU, higher is better

    Netra MT 1
    25.83
    Google Translate
    28.93

    Behind

  • ALT, English→Khmer

    1,018 professionally translated sentences

    NLLB-200 (1.3B) and MADLAD-400 also score higher

    COMET, higher is better

    Netra MT 1
    78.14
    Google Translate
    82.74

    Behind

  • ALT, Khmer→English

    1,018 professionally translated sentences

    The model's clear weakness: every working baseline beats it on every metric

    spBLEU, higher is better

    Netra MT 1
    19.47
    MADLAD-400 (3B)
    36.13

    Behind

AkaraAlpha

Full resultsLimitations

  • Held-out words

    5,714 words from the Khmer Text Transliteration Dataset

    Model alone, before the dictionary check

    CER %, lower is better

    AkaraAlpha
    15.07
    Transformer
    17.78

    Ahead

Sota

Full resultsLimitations

  • DDD test

    1,993 utterances of held-out Khmer speech

    sota-large (204M), pruned from that fine-tuned 1B

    CER %, lower is better

    Sota
    1.77
    Fine-tuned 1B (975M)
    2.47

    Ahead

  • FLEURS-km test

    771 utterances of read Khmer speech

    sota-large; in-domain, since FLEURS train is in the training mix

    CER %, lower is better

    Sota
    11.40
    Fine-tuned 1B (975M)
    9.44

    Behind

  • LibriSpeech test-clean

    2,620 utterances of English read from audiobooks

    sota-large; English is what pruning costs most

    WER %, lower is better

    Sota
    6.07
    Fine-tuned 1B (975M)
    3.61

    Behind