Publications

Abstract

We present Dhīr, a family of compact foundation models pretrained from random initialisation by Zenteiq for offline use on edge devices in industrial, network-denied and privacy-sensitive settings. Dhīr 2B is a dense transformer with 2.03 billion parameters, and Dhīr 5B a mixture-of-experts model with 5.09 billion parameters, 1.79 billion of them active per token; both use an in-house 131k-entry tokenizer for English and ten Indian languages. Every training stage, from random initialisation through more than six trillion tokens of pretraining to mid-training, supervised fine-tuning and reinforcement learning, ran on Google Cloud TPU v6e in India, with data and checkpoints held in the same region. We document the complete training stack, including a checkpoint lineage hashed from step 0 so that training from scratch can be verified. Quantised to 4 bits, Dhīr 2B is a 1.21 GB file that runs fully offline on a laptop’s integrated graphics and on an NVIDIA Jetson module.