NVIDIA, SLMs, and why small might just be the future of AI (again)

NVIDIA’s recent paper, “Small Language Models are the Future of Agentic AI,” dropped in June 2025 and immediately sparked conversations across X.

But if you’ve been following the space, or us, you’ll know we highlighted this trend In February 2025:

👉 Why companies are turning to small language models?

Now that the current is catching on, let’s break it down:

NVIDIA argues that Small Language Models (SLMs, <10B parameters) can rival or outperform larger models on narrow, repetitive tasks like intent classification, tool use, and structured response generation. They’re more efficient, faster to deploy, and significantly cheaper.

Instead of relying solely on monolithic LLMs (like GPT-4), NVIDIA suggests a heterogeneous architecture: let SLMs do the heavy lifting for routine subtasks and use LLMs selectively for broader context or deep reasoning.

Sound familiar?

That’s exactly the future we’ve been building toward at Pieces, with an even more radical angle: Nano Language Models (≤100M parameters) that run locally, on-device.

Nano > LLM? Let’s talk costs


Model Tier	Energy per Token	Source
Nano (≤100M)	1 mJ	GreenAI, Intel CPU tests
Cloud LLM (70B)	3–4 J	MIT SuperCloud benchmarks
Frontier LLM (~1T)	8–12 J	LLM Tracker survey
SLMs (1–34B)	0.04–2.0 J	Interpolated from MIT & Pieces labs

For a real-world comparison


Category	Cloud GPT‑4o	Pieces Nano Models (Local)	Savings
Energy Usage	3.65 kWh	0.001 kWh	↓ ~3.65 kWh
CO₂ Emissions	1.46 kg	0.0004 kg	↓ ~1.46 kg CO₂
Electricity Cost	$0.47	$0.00013	↓ ~$0.47
API Cost	$91.25	$0	↓ $91.25
Total Annual Cost	$91.72	~$0.00013	↓ ~$91.72

For teams? Multiply that


Team Size	Tokens/year	API $ Saved
50 Devs	0.18 B	$4,560
500 Devs	1.8 B	$45,600
5,000 Devs	18 B	$456,000

Some teams wait to adopt. Some are first in line

Looks like Sam won’t be alone, more teams are about to step into the fast fashion era of SaaS.