अ
Akshara
v0.5.0
Try it
Compare
Benchmarks
Evaluation
Rankings
Full results
Rankings
Every tokenizer, ranked on any test set
UTF-8 bytes per token on held-out text; higher means fewer tokens for the same text. Filter by provider or vocabulary size, or switch from single lines to whole documents.