The base: 3B active, built for agents

NVIDIA's own announcement positions Nemotron 3.5 Lightning as the small member of the Nemotron 3 family, aimed at the high-volume execution work an always-on agent spends most of its budget on: tool calls, output validation, subagent delegation. The official BF16 reference card gives the architecture in one line — "MoE — Mamba-2 + MoE + Attention hybrid" — 52 layers, 262,144-token native context with up to 1M supported on larger deployments, and 31,577,937,344 total parameters against roughly 3 billion active per token. Training ran on 20T+ tokens with a September 2025 pre-training cutoff; NVIDIA published the full suite under OpenMDW-1.1, including post-training data and recipes.

The agentic tilt matters here. NVIDIA's dev-blog numbers pair Lightning with NeMo Switchyard, a router that sends routine execution down to Lightning and escalates hard planning to frontier models, and the card's benchmark table is dominated by agent harnesses: SWE-bench Verified 51.56, Terminal-Bench 2.1 24.58, PinchBench 85.37, BrowseComp 36.97, τ³-bench (Banking) 9.28, plus GPQA Diamond 75.44 and MMLU Pro 81.94 on the knowledge side. Two weeks in, the base had 176,587 downloads and 190 likes — the fastest-moving NVIDIA open release in recent memory.

What a hybrid means for abliteration

Abliteration usually targets transformer checkpoints: probe activations on harmful vs harmless prompts, estimate the refusal direction, project it out of selected weight matrices. The Darkstar card's edit contract shows the same idea run on a model where most blocks are not transformers at all. The refusal direction was measured at layer 34 using 320 harmful and 320 harmless chat-templated prompts (mlabonne's harmful_behaviors plus harmless_alpaca), seed 42, and selected by a refusal-generation test that returned 0/8 across all 52 layers. The result is a single normalized harmful-minus-harmless unit vector — dimension 2688, norm 1.0000 — that gets projected out of the weights.

The target list is where the hybrid shows up. Across 3,126 residual-writing tensors the contract names: 2,944 routed-expert down-projections, 23 shared-expert down-projections, 6 attention output projections, 23 Mamba output projections, the MTP head's own o_proj and 128 expert down-projections, and the embedding weight. So the edit reaches into the state-space Mamba blocks and the sparse router outputs, not just the attention path. Everything is done in float32 shard-by-shard, W' = W − r(rᵀW), and re-rounded to BF16. The contract reports 3,126/3,126 tensors edited with a maximum normalized residual leakage of 0.000160 against a gate of 0.01 — meaning the refusal direction should be 62× fainter than the validation threshold — with the MTP head and all non-edited weights byte-identical to upstream.

What is proven here?

The publisher measured a behavior gate on the edited BF16 weights: 200/200 harmful compliance, 0/83 safe over-refusals, 0 errors (refusal-form marker set, temp 0, max_tokens 100). That is a real measurement, on a disclosed prompt set — not a claim that the model "never refuses" in general.

The NVFP4 twin: what stays protected

Darkstar publishes the edit as a four-cell family matrix: upstream BF16 control, clean ModelOpt NVFP4, abliterated BF16, and abliterated ModelOpt NVFP4. The quantized twin is the one worth running — a ~22 GB, 3-shard artifact that fits a single GPU. Its quantization contract is as explicit as the edit contract: 5,934 routed and shared expert up/down projections go W4A16-NVFP4 (group 16), while the parts that can't tolerate quantization stay in BF16 — the Mamba/SSM tensors (conv1d, in_proj, out_proj, A_log, D, dt_bias), the norms, embeddings, lm_head and MTP head.

Then it actually measures the result instead of assuming. On a single RTX PRO 6000 Blackwell with BF16 KV, the NVFP4 twin scores GPQA Diamond 141/198 = 71.2%, and the card is explicit about the gap to NVIDIA's 75.44: that delta is attributed to the serving stack (NVIDIA ran vLLM 0.26, FP8 KV, tensor-parallel 2, temp 1.0 averaged over eight repeats; Darkstar ran MTP10 + nemotron_v3 parser on one GPU), not to abliteration or quantization damage. Throughput with MTP10: 554.7 tok/s weighted across 4K/16K/48K contexts, 571.2 tok/s at 4K. Same 200/200 harmful compliance / 0/83 safe over-refusals gate on the quantized artifact.

How to run it

The BF16 edit is a standard vLLM-servable checkpoint, and the card's own serving line includes the reasoning parser and MTP speculative decoding:

vllm serve HangGlidersRule/Darkstar-Nemotron-3.5-Lightning-30B-A3B-Abliterated-BF16 \
  --max-model-len 131072 --kv-cache-dtype bfloat16 --reasoning-parser nemotron_v3 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":12}'

For a single consumer GPU, the NVFP4 twin (~22 GB) is the practical path, served with the same flags at 10 speculative tokens. No GGUF/Ollama page exists for the abliterated edit at the time of writing; the upstream NVIDIA checkpoint has an official ggml-org GGUF for local AI devices. Safety filtering is significantly reduced in this edit — that's the point. ABLITERATED.cloud's prepared profile pins revision f3723fc56c3e05bf8a9499b985dec8cced37027c at an approximate managed price estimate of $5.45/hour (1 × H200 profile for the 30B-A3B class).

The creator: HangGlidersRule

The edit comes from the Hugging Face publisher HangGlidersRule, a PRO account that also runs the model-forge repository on GitHub — "reproducible model transformation, quantization, serving, evaluation, and Darkstar release catalog." The GitHub profile names the operator "bob" in Washington DC, an account dating to 2011 with eight public repos, and the forge documents a working method: pinned recipes, fail-closed validators, sha256 manifests, and a deliberate separation between public model records and private operation archives. The same Darkstar brand shipped a Qwen3.8-27B abliteration family in late August (BF16 edit plus an NVFP4-Mixed-FP8 quant, 646 downloads on the flagship). The base model itself is NVIDIA's, released with full weights, data and recipes under OpenMDW-1.1 — so the lineage here is NVIDIA upstream, Darkstar edit, both licenses retained.

The idea, in plain words

A state-space block is a way to remember without a cache

Normal transformers replay the whole past conversation for every new token — powerful, but memory-hungry. A Mamba-2 state-space block instead compresses history into a fixed-size internal state and updates it as it goes, which is faster and uses far less memory per token. Nemotron 3.5 Lightning interleaves those state-space blocks with classic mixture-of-experts layers: ~3B of its 30B parameters fire per token, giving a small model's speed with a much larger model's knowledge. Abliterating that hybrid means removing the refusal direction from the Mamba output projections and expert down-projections too — the edit touches every kind of residual writer the architecture has.

Primary sources