NVIDIA Nemotron 3.5 Explained: Open Agent Models, and 5 Tools to Run Them

On October 15, NVIDIA expanded its open Nemotron model family with three new releases aimed squarely at AI agents — from enterprise data centers down to the laptop on your desk. The headline is the kind of thing that would have sounded absurd a year ago: a 74-billion-parameter model, squeezed by native 2-bit quantization into 24 GB of memory, built to run agentic workloads locally without ever touching the cloud.

The release matters for a simple reason. Closed frontier models keep getting smarter, but they also keep your data and your bills in someone else's data center. Nemotron 3.5 is NVIDIA betting that the next phase of the AI race is about where intelligence runs — and that a meaningful share of it will run on hardware you own.

What NVIDIA announced

The October 15 launch (detailed in NVIDIA's official blog post) covers three models:

• Nemotron 3.5 Super VL — a 120-billion-parameter vision-language model (12B active per token, MoE) for specialized always-on agents. It handles text, images, charts, web pages, and PDFs, and NVIDIA positions it for multistep enterprise workflows like document review and tool-heavy research.
• Nemotron 3.5 Bolt 74B-A9B 24GB — the local hero: 74B total parameters with 9B active per token, compressed with native 2-bit quantization-aware training to fit in 24 GB of memory. Built for web research, writing, and code debugging on RTX Spark laptops, GeForce RTX PCs, DGX Spark, DGX Station, and RTX PRO workstations.
• Nemotron 3.5 Bolt E4B 4GB — a compact edge model for fast conversational agents, voice assistants, and interactive characters on GeForce RTX, Jetson, and Spark systems. It arrives October 23.

All three ship with open weights, open datasets, and training recipes. NVIDIA says organizations can post-train the models on their own data, deploy on their own infrastructure, and customize through NVIDIA NeMo.

One caution on the numbers: NVIDIA claims Super VL delivers up to 5x faster throughput than the next best open model in its class, citing the Artificial Analysis Intelligence Index, and calls Bolt 74B the #1 open model in its class on PinchBench. Those are the company's own benchmarks — useful signal, not independent verification. As independent analysts at Omdia noted in TechTarget's coverage, NVIDIA has an obvious self-interest here: open models that run best on NVIDIA GPUs reinforce the ecosystem around its hardware.

Five tools that can run these models today

NVIDIA's announcement names the launch-day runtimes explicitly: Bolt 74B-A9B is available on Hugging Face with support for llama.cpp, Ollama, LM Studio, Unsloth, and vLLM — and runs directly in Hermes and OpenClaw. Here is what each one does, and which part of the Nemotron story it fits.

1. Ollama

Ollama is the simplest on-ramp to local models: a command-line tool that downloads and runs open-weight models on your own machine. If you want to try Bolt 74B without learning a new stack, Ollama is the path of least resistance — pull the model, run it, and your agent workloads stay on your hardware.

2. LM Studio

LM Studio is a desktop app for discovering, downloading, and chatting with local LLMs, with a polished GUI instead of a terminal. For Nemotron 3.5, that means a point-and-click way to load the 24 GB Bolt build, inspect its behavior, and serve it to other apps over a local API — no cloud account required.

3. llama.cpp

llama.cpp is the open-source C/C++ inference engine that quietly powers most of the local-AI ecosystem. It has no dependencies, runs on everything from Apple silicon (Metal) to NVIDIA GPUs (CUDA) to plain CPUs, and supports 1.5-bit through 8-bit quantization — which is exactly the territory Bolt 74B's native 2-bit build lives in. Its llama serve command exposes an OpenAI-compatible API, so existing agent harnesses can point at it with minimal changes.

4. Unsloth

Unsloth is an open-source framework and desktop app for running and training models locally — the only tool on this list that also lets you fine-tune. Its unsloth start command connects coding agents like Claude Code and Codex to your local models, and its docs already carry a Nemotron 3.5 Bolt page, so day-zero support is real. If your plan is to adapt Super VL or Bolt to your own data rather than just run them, start here.

5. vLLM

vLLM is the high-throughput serving engine for production deployments. Where the others optimize for a single machine, vLLM is built to serve models to many users at once with low latency — the right choice if you're putting Super VL behind an API for a team or a product, especially on NVIDIA's own serving stack.

Who should pay attention

• Developers building agents: Bolt 74B gives you an open base model tuned for tool use and multistep work that fits a single high-end workstation — no per-token bill, no data leaving the building.
• Enterprises with data-residency requirements: Super VL's open weights plus on-prem deployment is the pitch for regulated industries. Per NVIDIA, partners from Scale AI to CrowdStrike are already post-training it for finance, security, and document workflows.
• Hardware enthusiasts: if you own an RTX 5090-class machine or are eyeing an RTX Spark laptop, Bolt 74B is the first 74B-class agent model that realistically fits your VRAM.

Strengths and limits, honestly

The strengths are real: genuinely open weights, datasets, and recipes; a model tier for every deployment size; and day-one support across the local-AI tooling ecosystem. The 2-bit quantization story is the most interesting technical bet — if quality holds up, it resets expectations for what fits on consumer hardware.

The limits deserve equal airtime. Every benchmark in NVIDIA's announcement is vendor-reported; independent testing on agentic tasks, tool-use accuracy, and long-context behavior hasn't landed yet. The 2-bit diet is aggressive, and as independent analysts have noted, NVIDIA hasn't shown what that compression costs against the uncompressed parent model. And "runs anywhere" still means NVIDIA hardware in practice — the local story is built around RTX and Spark silicon. Pricing for hosted access via NIM and cloud partners is also still taking shape, so teams comparing against per-token cloud APIs can't do a clean cost comparison yet.

How to get started

1. Pick your tier: Super VL for multimodal enterprise agents, Bolt 74B for local agentic work, Bolt E4B (from Oct 23) for edge and conversational use.
2. Download the weights from Hugging Face — NVIDIA publishes them openly, and Super VL is also available via OpenRouter, ModelScope, and NVIDIA NIM.
3. Choose a runtime from the five above: Ollama or LM Studio for the fastest start, llama.cpp for maximum control, Unsloth if you plan to fine-tune, vLLM for serving at scale.
4. Test before you commit: run your actual agent workloads — tool calls, document parsing, multistep tasks — and compare against the cloud models you're using today. Vendor benchmarks are a starting point, not a verdict.
5. Watch the ecosystem: with weights, datasets, and training recipes all open, expect community fine-tunes and quantized variants to appear within days — the open-model flywheel tends to move fast after an NVIDIA release.

The bottom line

Nemotron 3.5 isn't really a story about one model beating another on a leaderboard. It's NVIDIA's clearest statement yet that the open-model strategy is about deployment: give every tier of hardware an open agent model, from the data center to the desktop to the edge, and let the ecosystem — Ollama, LM Studio, llama.cpp, Unsloth, vLLM — do the rest. Whether 2-bit local agents become the default way to build depends on what independent testing shows in the coming weeks. But the direction is unmistakable: the frontier is moving onto your machine.

Sources

• NVIDIA Blog: "Open From Data Center to Desktop: NVIDIA Expands Nemotron Family With Highest Efficiency Super VL and Bolt Models" (published Oct 15, 2026)
• TechTarget: "Open source and on device: the strategy behind Nvidia's new AI models" (Oct 16, 2026)
• llama.cpp on GitHub — project README and documentation
• Unsloth Docs — platform support, features, and model coverage