Sinhala Grammar Checker
A specialized LoRA fine-tune of SinLlama designed for Sinhala journalism. Engineered to fix real editorial errors without rewriting the journalist's sentence—preserving meaning and stylistic intent.
Accuracy
87.7%
Stage 2 Overall Accuracy
Over-Correction
6.7%
Strict exact string match
Training Data
36K+
Hand-curated examples
Adapter Version
v22
Current production deployment
Training & Architecture Details
LoRA Configuration
Trained via Unsloth and TRL SFTTrainer, targeting both attention and MLP layers (q, k, v, o_proj + gate, up, down_proj) with rank r=32. Applying LoRA to MLP layers proved crucial for resolving lexical bugs (e.g., verb-stem selection) that attention-only adapters missed.
- Loss Function: Completion-only loss ensures the model focuses entirely on the corrected response tokens, preventing wasted gradient steps on the instruction text.
- Compute Profile: 5 epochs, effective batch size of 8 (2 × 4 grad-accum), lr 5e-5 cosine schedule. Requires ~1h20m on a single NVIDIA A40 GPU.
Dataset Engineering
The model relies on 36,006 rows of highly curated, deduplicated data (cleaned_v9_full.jsonl). The dataset is composed of strictly isolated, rule-specific files to prevent regressions.
- Spelling & Confusables (ණ/න, ළ/ල)
- Already-Correct Sentences (Control set)
- Subject-Object-Verb (SOV) order
- Literary Plural Verb Agreement
Note: The changed/unchanged ratio is held strictly near 65/35 to prevent the model from always assuming an edit is required, which drives down over-correction rates.