Token costs are climbing for enterprises engaging with AI technologies, and developers are at the forefront of addressing this financial challenge. At the recent DevSparks 2026 summit in Bengaluru, Jigar Halani of NVIDIA South Asia highlighted the critical role developers play in managing and optimizing token economics, which has become a pressing concern in boardrooms worldwide.
### Model Selection: Balancing Cost and Capability
Developers have a significant impact on AI costs through their choices of model size and type. The decision between various model sizes—ranging from 30 billion to a trillion parameters—directly affects throughput, latency, and, ultimately, cost. Halani emphasized that larger models are not always better, especially for simple workflows that can operate efficiently on smaller models. This approach mirrors past software development practices where optimizing for limited server memory was crucial.
Related Posts
The concept of context length also plays a crucial role in token consumption. Larger context lengths in AI models lead to increased token usage, especially in high-frequency workflows. Developers must therefore apply discipline in managing token consumption, akin to optimizing code for server resources in traditional programming environments.
### The Growing Demand for Tokens
Token demand has surged dramatically, with non-tech companies moving from consuming millions to billions of tokens annually, and tech companies reaching trillions. This increase is driven by the shift from conversational to agentic AI, which significantly raises token usage. For example, a simple request for an AI agent to book a flight can generate far more tokens than a basic conversational interaction.
The decision between using open-source models versus proprietary APIs is also becoming more critical. As the performance gap between these options narrows, organizations must weigh the benefits of owning their model infrastructure against the convenience and initial cost-efficiency of API access. For enterprises with extensive AI usage across multiple teams, owning infrastructure may become more cost-effective than continuing with per-token API charges.
### Implications for India’s Startup Ecosystem
The rising cost of tokens presents both a challenge and an opportunity for India’s burgeoning startup ecosystem. Startups that can develop efficient AI models and offer solutions for optimizing token usage stand to gain a competitive edge. This is especially relevant as Indian startups increasingly integrate AI into their products and services, necessitating a balance between technological advancement and cost management.
Developers in India have the chance to shape the future of AI by focusing on model efficiency and token economics. Their expertise can drive innovation in AI applications while keeping operational costs in check, a critical factor for startups operating with limited budgets.
As token usage continues to grow, developers, founders, and investors should closely monitor advancements in AI model efficiency and infrastructure ownership. The ability to optimize token consumption will be a key differentiator in the competitive landscape, influencing investment decisions and the strategic direction of AI-driven startups in India.
