← Blog
Project

Netra-OCR

A Squeeze-and-Excitation Transformer network for Khmer optical character recognition

Parameters 17M
Training images 1.3M
License MIT
GitHubModel weightsDatasetLive demo
Bar chart comparing character error rates across six models on three benchmarks
Figure 1 — Character Error Rate (%) across all evaluation datasets, benchmarked against five models. Qwen2.5-VL (3B) and DeepSeek-OCR (3B) were fine-tuned on the same training set before evaluation. Lower is better.

Abstract#

Netra-OCR is a 17M-parameter model designed to process variable-length text-line images with high accuracy and low latency. Trained from scratch on a dataset of 1.3M bilingual (Khmer and English) text-line images, the model employs an encoder-decoder architecture. The vision encoder integrates a Squeeze-and-Excitation (SE) network with a Transformer encoder to extract robust spatial features, while the decoder uses a standard Transformer architecture for autoregressive text generation.

To accommodate the distinct orthographic structures of the two languages, we implement a hybrid tokenization strategy: English text is processed strictly at the character level, whereas Khmer text uses a mixture of character-level and character-cluster representations.

Datasets#

The model was trained entirely on synthetic data and evaluated on real-world and synthetic datasets.

Sample text-line images from each of the training and evaluation datasets
Figure 2 — Overview of the training and evaluation data, showing representative text-line images from each source.

Training data (synthetic)#

We generated 200,000 synthetic text-line images across 11 typographical fonts, and combined them with two external open-source datasets — SoyVitou/KhmerSynthetic1M and seanghay/khmer-hanuman-100k — for 1.3M training images in total.

DatasetCountSource
Custom document text100,000Pillow + Khmer corpus, with erosion, noise, and perspective distortion
Custom scene text100,000SynthTIGER + Stanford BG, with rotation, blur, and realistic backgrounds
KhmerSynthetic1M1,000,000External (SoyVitou)
khmer-hanuman-100k100,000External (seanghay)

Evaluation data (real-world + synthetic)#

Evaluation combines the public Khmer OCR Benchmark (KHOB) with a proprietary, custom-annotated set of smartphone-captured legal documents — birth certificates, diplomas, and national ID cards — to test real-world applicability, plus a synthetic printed-word set to assess short-sequence recognition specifically.

DatasetTypeSizeDescription
KHOBReal325Standard benchmark, clean backgrounds but compression artifacts.
Legal DocumentReal227High variation in degradation, illumination, and distortion.
Printed WordsSynthetic1,000Short, isolated words in 10 different fonts.

Architecture#

Netra-OCR uses a "chunk-and-merge" strategy to handle variable-length lines without aggressive resizing: input images are resized to a fixed height of 48px, then split into overlapping 48×100px chunks (16px overlap).

Diagram of the Netra-OCR encoder-decoder pipeline from input image through chunking, SE network, transformer encoder, merging, BiLSTM and transformer decoder
Figure 3 — The input image is resized and split into 48×100px chunks with 16px overlaps. Each chunk is processed in parallel by the Squeeze-and-Excitation network, producing 512 feature maps of 2×32px. These become patch embeddings with positional embeddings; the Transformer encoder turns them into vision tokens, which are merged, smoothed by a bidirectional LSTM, and decoded into character clusters.

Each chunk passes through a Squeeze-and-Excitation backbone — five VGG-style convolutional blocks with height-only pooling in later stages, so the width axis (where character order lives) is preserved. A 1D SE module then recalibrates the feature maps per horizontal column rather than globally, letting the network weight noisy channels independently at each position along the line.

Diagram of the squeeze, excitation and scale steps applied per horizontal column
Figure 4 — Unlike a standard SE block, the squeeze averages across the height axis only, producing a per-column channel descriptor rather than one global descriptor. The excitation gate reduces and restores the channel dimension through a bottleneck, and the scale step reweights the feature map column by column.

The resulting feature map is projected into a sequence of patch embeddings and passed through a Transformer encoder, producing per-chunk "vision tokens." Because a line is split into overlapping chunks encoded independently, the seams between them can be inconsistent — so after merging all chunks back into one sequence, a bidirectional LSTM smooths the representation across chunk boundaries before a standard autoregressive Transformer decoder generates the output character-cluster sequence.

Diagram of a bidirectional LSTM sweeping forward and backward across merged chunk boundaries
Figure 5 — A single bidirectional LSTM layer sweeps the merged sequence in both directions, so each token's representation is informed by tokens on both sides of any chunk seam — resolving the discontinuity when a character is split across two independently-encoded chunks.

Results#

Netra-OCR was benchmarked against five models, including vision-language models such as Qwen2.5-VL (3B) and DeepSeek-OCR (3B) fine-tuned on the same training set, measured by Character Error Rate (CER, lower is better) across KHOB, legal documents, and printed-word benchmarks.

Sample document images with the model's recognized Khmer text alongside
Figure 6 — Recognized samples from the evaluation set.
Sample document images with the model's recognized Khmer text alongside
Figure 7 — Recognized samples from the evaluation set.

References#

  1. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, et al.
    ICLR 2021.
    arXiv:2010.11929

  2. TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models
    Minghao Li, Tengchao Lv, Lei Cui, Yijuan Lu, Dinei Florencio, Cha Zhang, Zhoujun Li, Furu Wei.
    AAAI 2023.
    arXiv:2109.10282

  3. Toward a Low-Resource Non-Latin-Complete Baseline: An Exploration of Khmer Optical Character Recognition
    R. Buoy, M. Iwamura, S. Srun and K. Kise.
    IEEE Access, vol. 11, pp. 128044-128060, 2023.
    DOI: 10.1109/ACCESS.2023.3332361

  4. Balraj98. (2018). Stanford background dataset [Data set]. Kaggle. https://www.kaggle.com/datasets/balraj98/stanford-background-dataset

  5. EKYC Solutions. (n.d.). Khmer OCR benchmark dataset (KHOB) [Data set]. GitHub. https://github.com/EKYCSolutions/khmer-ocr-benchmark-dataset

  6. Em, H., Valy, D., Gosselin, B., & Kong, P. (2024). Khmer text recognition dataset [Data set]. Kaggle. https://www.kaggle.com/datasets/emhengly/khmer-text-recognition-dataset

  7. Squeeze-and-Excitation Networks
    Jie Hu, Li Shen, and Gang Sun.
    CVPR 2018.
    arXiv:1709.01507

  8. Bidirectional Recurrent Neural Networks
    Mike Schuster and Kuldip K. Paliwal.
    IEEE Transactions on Signal Processing, 1997.
    DOI: 10.1109/78.650093