Skip to content

Experimental Models: Command R7B, Aya Expanse 8B, SmolLM3-3B

CircuitKIT's TransformerLens compatibility port (circuitkit.backends._tl_compat) started with Tiny Aya (cohere2) and has since grown into a multi-architecture seam: one internal registry (_tl_compat/registry.py) that lets several HF architectures share the same convert_hf_model_config / get_pretrained_state_dict dispatch and a per-architecture, per-layer rotary (NoPE) policy. Three more model families now ride that seam:

Model HF architecture model_type Port module
Tiny Aya Cohere2ForCausalLM cohere2 _tl_compat/cohere.py
Command R7B Cohere2ForCausalLM cohere2 _tl_compat/cohere.py
Aya Expanse 8B CohereForCausalLM cohere _tl_compat/cohere.py
Aya Expanse 32B CohereForCausalLM cohere _tl_compat/cohere.py
SmolLM3-3B SmolLM3ForCausalLM smollm3 _tl_compat/smollm3.py

Aya Expanse 32B shares Aya Expanse 8B's config/weight converter unchanged — registration, real-weight config conversion, and module resolution for pruning score extraction and quantization targets are validated on the real checkpoint (see Tests). Full-depth bf16 parity, discovery, evaluation, and weight steering do not run yet. A loaded TransformerLens copy needs about 64 GiB for weights alone, so it does not fit on most single GPUs, and multi-GPU loading through n_devices is affected by a bug in transformer-lens==3.8.0 (see the warning in Tests). The other four models on this page are experimental discovery/evaluation/intervention targets: TransformerLens 3.8.0 has no native support for any of cohere, cohere2, or smollm3; the port adds it additively, at import time, with every patch falling through to stock TL for any other architecture. Command R7B and Aya Expanse are not gated on the Hub the way tiny-aya is, but both are flagged gated=auto — an account needs to accept Cohere's license click-through once before the safetensors are downloadable, using the same HF_TOKEN / cached-login auth path as tiny-aya. SmolLM3-3B is a fully public repository — no HF_TOKEN needed.

Experimental

These are experimental discovery targets. Results should only be trusted once the real-weight parity gate passes on your machine (see Tests below).

Local checkpoint folders

While CircuitKIT's in-memory TransformerLens patch series is active (transformer-lens==3.8.0, with import circuitkit before the first import transformer_lens), a local checkpoint folder loads from its config.json whatever it is called (a fine-tune in meta-llama-Llama-3.2-1B_sft/ is not mistaken for base Llama-3.2-1B). Without the series, a folder whose path contains "llama" or "gemma" is refused. Local Gemma checkpoints, fine-tunes included, are refused until CircuitKIT moves to TransformerLens 4: TransformerLens 3.8 has Gemma configs for Hub names only, and loading the Hub ID gives the base weights. Local SmolLM3 checkpoint folders are not supported by the current port; load HuggingFaceTB/SmolLM3-3B by Hub ID.

Gemma-4 and Sarvam-MoE: discovery-only TransformerLens ports

The Gemma-4 and Sarvam-MoE patches provide a discovery-only TransformerLens path. They do not establish support for CircuitKIT evaluation, pruning, quantization, steering or interventions; those surfaces are not validated for these models. Treat successful loading as experimental, not as full model support.

Checkpoint Scope and requirements
google/gemma-4-31B-it Text-side discovery only; the multimodal vision tower is not exposed through the TransformerLens port. At bf16, weights alone are about 58 GiB; use an 80 GB-class GPU or larger to leave room for activations.
sarvamai/sarvam-30b TransformerLens discovery only. Sarvam ships its modeling code in the repository rather than in transformers, so it needs trust_remote_code: load_model(..., trust_remote_code=True), trust_remote_code: true under model: in a YAML/dict config, or circuitkit discover --trust-remote-code. This executes code from the model repository, so it is off by default and never inferred. At bf16, weights alone are about 56 GiB; use an 80 GB-class GPU or larger to leave room for activations.

Supported checkpoints

  • CohereLabs/c4ai-command-r7b-12-2024 (Command R7B, cohere2)
  • CohereLabs/aya-expanse-8b (Aya Expanse 8B, cohere1/cohere)
  • CohereLabs/aya-expanse-32b (Aya Expanse 32B, cohere1/cohere) — registration, config conversion, truncated-depth parity and pruning / quantization module resolution validated on real weights; full-depth parity, discovery, evaluation and weight steering do not run yet (see Tests)
  • HuggingFaceTB/SmolLM3-3B (SmolLM3-3B, smollm3)

Architecture at a glance

Property Command R7B Aya Expanse 8B Aya Expanse 32B SmolLM3-3B
HF architecture Cohere2ForCausalLM CohereForCausalLM CohereForCausalLM SmolLM3ForCausalLM
Params ~7B ~8B ~32B ~3B
d_model / layers 4096 / 32 4096 / 32 8192 / 40 2048 / 36
Query / KV heads 32 / 8 (GQA, 4:1) 32 / 8 (GQA, 4:1) 64 / 8 (GQA, 8:1) 16 / 4 (GQA)
d_head 128 128 128 (not an explicit config field; hidden_size // n_heads) 128
d_mlp (gated SiLU) 14336 14336 24576 11008
Normalization LayerNorm (bias-less) LayerNorm (bias-less) LayerNorm (bias-less) RMSNorm
Block topology parallel attn+MLP, one input LN parallel attn+MLP, one input LN parallel attn+MLP, one input LN sequential (standard Llama)
RoPE style GPT-J interleaved (rotary_adjacent_pairs=True) interleaved (rotary_adjacent_pairs=True) interleaved (rotary_adjacent_pairs=True) standard Llama, non-interleaved (rotary_adjacent_pairs=False)
rope_theta 50000 10000 4,000,000 5000000
Attention pattern sliding-window (4096) ⨯ full, interleaved all full attention all full attention all full attention
NoPE (no rotary) layers full-attention layers none — rotary everywhere none — rotary everywhere per-layer, driven by config's no_rope_layers (every 4th layer on the real checkpoint)
logit_scale 0.25 (real fold) 0.125 (real fold) 0.0625 (real fold) none (no post-multiply at all)
Tied embeddings yes yes yes yes
Vocab 256000 256000 256000 128256
Max position embeddings (real ckpt) 132096 (preserved) 8192 8192 65536 (preserved)
Checkpoint size — ~15 GiB (fp16, 4 shards) ~60.2 GiB (fp16, 14 shards) —
Gated on the Hub? gated=auto (license click-through) gated=auto (license click-through) gated=auto (license click-through) fully public

Aya Expanse 32B's tokenizer is byte-identical to 8B's (confirmed via tokenizer.json SHA-256, both starting c69a7ea6c0927dfa), so the existing single-token task guards (greater_than's own operand pool, ioi's name check) behave the same way they do for 8B.

For reference, Tiny Aya is 36 layers / d_model=2048 / 16:4 GQA heads / d_mlp=11008, the same cohere2 shape as Command R7B but with logit_scale=1.0 (a no-op fold) and a smaller, ~3.35B-parameter checkpoint.


The _tl_compat seam

All four models' support lives in circuitkit.backends._tl_compat, applied at import time from circuitkit/backends/__init__.py:

from . import _tl_compat
_tl_compat.apply_patches()
  • _tl_compat/registry.py — the shared multi-arch seam: an ArchPort dataclass (architecture, model_names, config_converter, weight_converter, should_skip_rotary) registered per HF architecture, resolved by model name or by architecture at call time. apply_rotary is patched once per AbstractAttention subclass and dispatches to whichever port's NoPE policy applies — a policy takes (cfg, layer_id, layer_attn_type) so it can key off either the layer index (SmolLM3's per-layer mask) or TL's local/global attn-type string (cohere2's sliding-window split).
  • _tl_compat/cohere.py — Tiny Aya, Command R7B (cohere2), and Aya Expanse (cohere1/CohereForCausalLM). cohere1 and cohere2 share the exact same weight converter (convert_cohere2_weights); only the config converter and NoPE policy differ.
  • _tl_compat/smollm3.py — SmolLM3-3B, a structurally distinct Llama-family port (RMSNorm, sequential block, standard rotary, per-layer NoPE) with its own weight converter.

Config mapping (convert_hf_model_config)

Command R7B — identical mapping to Tiny Aya (same cohere2 converter); the only difference is logit_scale=0.25 instead of 1.0:

HF / spec TL HookedTransformerConfig
rope_gptj (interleaved pairs) rotary_adjacent_pairs=True
rope_theta=50000 rotary_base=50000
sliding_window=4096 + layer_types use_local_attn=True, window_size=4096, attn_types
parallel block, single input_layernorm parallel_attn_mlp=True
num_key_value_heads n_key_value_heads (GQA)
LayerNorm normalization_type="LN"
logit_scale=0.25 folded into unembed.W_U at weight-conversion time

Aya Expanse 8B — a simplification of the cohere2 mapping above, via its own _convert_cohere1_config (cohere1's HF config has no sliding_window/layer_types attributes at all, and doesn't always set an explicit head_dim, so this is a separate function rather than a branch):

HF / spec TL HookedTransformerConfig
rotate_half interleaved (confirmed from modeling_cohere.py source) rotary_adjacent_pairs=True
rope_theta=10000 rotary_base=10000
no sliding_window / layer_types at all use_local_attn=False, no attn_types — rotary and full attention on every layer
parallel block, single input_layernorm parallel_attn_mlp=True
head_dim not always set getattr(hf_config, "head_dim", None) or (hidden_size // n_heads)
logit_scale=0.125 folded into unembed.W_U
use_qk_norm=True (unused today) raises NotImplementedError — the weight converter has no q/k-norm conversion path

Aya Expanse 32B uses the exact same _convert_cohere1_config function and convert_cohere2_weights weight converter as 8B — no new code, only a second repo ID in AYA_EXPANSE_MODEL_NAMES. Only the config values differ: rope_theta=4,000,000, logit_scale=0.0625, 64/8 GQA heads, d_mlp=24576, 40 layers. Validated on real weights at a truncated depth (see Tests); the mapping table above is identical in structure to 8B's, just with 32B's numbers.

SmolLM3-3B — a new Llama-family mapping, via _convert_smollm3_config:

HF / spec TL HookedTransformerConfig
RMSNorm normalization_type="RMS", final_rms=True, eps=rms_norm_eps
sequential decoder block parallel_attn_mlp=False
standard (non-interleaved) Llama rotary (rotate_half splits first/second half) rotary_adjacent_pairs=False
rope_theta=5000000 rotary_base=5000000
no_rope_layers list (1=apply rotary, 0=skip) per-layer NoPE policy, see below
gated SiLU MLP gated_mlp=True, act_fn="silu"
no logit scale unembed.W_U is an unscaled transpose of lm_head.weight
head_dim not always set same getattr(..., None) or (hidden_size // n_heads) fallback as cohere1
attention_bias / mlp_bias / use_sliding_window = True each raises NotImplementedError — none true on the real checkpoint, but the converter would otherwise silently produce a wrong model

Context length and memory

The ports preserve each checkpoint's full max_position_embeddings in TransformerLens 3.8. That release keeps an empty causal-mask buffer and builds the mask for the active input length at forward time; it does not allocate a dense n_ctx × n_ctx mask during model construction. Longer input sequences still require more attention memory, so choose sequence length and batch size for the available device rather than relying on an artificial 8192-token cap.

Weight conversion

Command R7B and Aya Expanse share convert_cohere2_weights (see Tiny Aya for the GQA / parallel-block / tied-embedding details, which apply unchanged). The one thing that varies per checkpoint is the logit_scale fold: unembed.W_U = lm_head.weight.T * logit_scale — 0.25 for Command R7B, 0.125 for Aya Expanse, 1.0 (a no-op) for Tiny Aya.

SmolLM3-3B has its own convert_smollm3_weights, mirroring TL's own convert_llama_weights: RMSNorm .w-only (no bias tensors anywhere — unlike the cohere family's explicit zero-bias trick, RMSNorm has no bias term to satisfy in the first place), _W_K/_W_V GQA slots, gated-SiLU W_gate/W_in/W_out, tied W_E/W_U with no scale fold.

All zero-tensor constructions (biases, LN zero-biases where applicable) are built on next(model.parameters()).device rather than a bare torch.zeros(...) (which silently defaults to CPU) — this matters once the source HF model is loaded explicitly onto a GPU, as the real-weight parity gates do.

NoPE

  • Command R7B (same policy as Tiny Aya): Cohere2Attention.forward only rotates q/k when self.sliding_window is not None, so periodic full-attention ("global") layers get no positional embedding at all.
  • Aya Expanse: CohereAttention.forward calls apply_rotary_pos_emb unconditionally on every layer — there is no per-layer branch in this architecture at all, so the registered NoPE policy always returns False.
  • SmolLM3-3B: a genuinely per-layer policy. HF's no_rope_layers list means truthy (1) = apply rotary, falsy (0) = skip (confirmed from SmolLM3Attention.__init__'s self.use_rope = config.no_rope_layers[layer_idx]). Because HookedTransformerConfig is a plain dataclass with no "extra data" slot, the resolved per-layer list can't be threaded through the config dict the way cohere2 reuses TL's own attn_types field for its local/global split — it is instead stashed in a small module-level cache keyed by official_model_name (the same string already written to cfg.tokenizer_name), and the registered rotary policy looks it up from there at call time.

Running discovery

export HF_TOKEN=hf_...   # Command R7B / Aya Expanse only (license click-through); not needed for SmolLM3
import circuitkit as ck

model = ck.load_model("CohereLabs/c4ai-command-r7b-12-2024")   # or aya-expanse-8b / SmolLM3-3B
circuit = ck.discover(model, task="greater_than", algorithm="eap-ig")

Task caveat — single-token answers

Same caveat as Tiny Aya: the single-answer-position discovery metrics assume each answer is one token. greater_than builds its own single-token operand pool against the loaded tokenizer and is the recommended smoke task for all three models. ioi's 99-name pool was verified to stay single-token under Command R7B's 256k vocab (a larger vocab than GPT-2's 50k only shrinks the risk of a name splitting, never grows it); it is used as the ACDC opt-in task below since ACDC has no built-in greater_than support.


Tests

Offline unit tests (no network, no GPU, no gated weights) run in ordinary CI:

  • tests/backends/tl_compat/test_command_r7b.py — registration, config mapping (including the logit_scale=0.25 fold and the preserved context length).
  • tests/backends/tl_compat/test_aya_expanse.py — registration, the cohere1 config converter (no sliding window / attn_types, head_dim fallback, use_qk_norm guard), rotary-everywhere NoPE policy.
  • tests/backends/tl_compat/test_aya_expanse_32b.py — the same coverage at 32B's real spec (40 layers, 8:1 GQA, rope_theta=4e6, logit_scale=0.0625), plus a tiny random HF model built with those specific values matching HookedTransformer logits to 1e-5.
  • tests/backends/tl_compat/test_smollm3.py (41 cases) — registration, config/weight conversion, per-layer NoPE (TestSmolLM3PerLayerNoPE, parametrized across NoPE and non-NoPE layer indices) and RMSNorm-vs-LayerNorm (TestSmolLM3RmsNormNotLayerNorm) assertions.
  • tests/apply/test_architecture_registry.py — offline arch_registry resolution/mock coverage for the cohere and smollm3 families (MockCohereModel, MockSmolLM3Model).

Real-weight gates are opt-in, one env var per family, and are slow. Each family has four gates: parity (the numerical proof), discovery (end-to-end circuit finding), evaluation (faithfulness), and interventions (pruning/quantization/steering module resolution) — ~20 Group F test files across all four models in this port.

Command R7B (CIRCUITKIT_RUN_COMMAND_R7B=1, needs HF_TOKEN):

CIRCUITKIT_RUN_COMMAND_R7B=1 HF_TOKEN=... python -m pytest \
    tests/backends/tl_compat/test_command_r7b_parity.py \
    tests/regression/test_command_r7b_discovery.py \
    tests/regression/test_command_r7b_evaluation.py \
    tests/regression/test_command_r7b_interventions.py -v

# ACDC only (separate opt-in — see "ACDC is excluded" below):
CIRCUITKIT_RUN_COMMAND_R7B=1 CIRCUITKIT_RUN_COMMAND_R7B_ACDC=1 HF_TOKEN=... \
    python -m pytest tests/regression/test_command_r7b_discovery.py -v -k acdc

Aya Expanse 8B (CIRCUITKIT_RUN_AYA_EXPANSE=1, needs HF_TOKEN):

CIRCUITKIT_RUN_AYA_EXPANSE=1 HF_TOKEN=... python -m pytest \
    tests/backends/tl_compat/test_aya_expanse_parity.py \
    tests/regression/test_aya_expanse_discovery.py \
    tests/regression/test_aya_expanse_evaluation.py \
    tests/regression/test_aya_expanse_interventions.py -v

# ACDC only:
CIRCUITKIT_RUN_AYA_EXPANSE=1 CIRCUITKIT_RUN_AYA_EXPANSE_ACDC=1 HF_TOKEN=... \
    python -m pytest tests/regression/test_aya_expanse_discovery.py -v -k acdc

Aya Expanse 32B (CIRCUITKIT_RUN_AYA_EXPANSE_32B=1, needs HF_TOKEN, deliberately a separate flag from 8B's so an 8B opt-in never silently triggers a 60 GiB download):

CIRCUITKIT_RUN_AYA_EXPANSE_32B=1 HF_TOKEN=... python -m pytest \
    tests/backends/tl_compat/test_aya_expanse_32b_parity.py \
    tests/regression/test_aya_expanse_32b_interventions.py -v

This runs what passes today: truncated-depth parity and the pruning / quantization module-resolution checks. Everything that needs the full 40-layer model through TransformerLens (full-depth parity, discovery, evaluation, weight steering) is skipped unless CIRCUITKIT_RUN_AYA_EXPANSE_32B_FULL=1 is also set, because it fails on transformer-lens 3.8.0 (see below).

32B: truncated-depth parity and pruning/quantization resolution pass; full-depth parity, discovery, evaluation, and weight steering are blocked by a transformer-lens bug

Validated against the real checkpoint:

  • Registration — the repo ID resolves to the same cohere1 ArchPort as 8B.
  • Config conversion at 32B's real spec (40 layers, 8:1 GQA, rope_theta=4e6, logit_scale=0.0625).
  • Truncated-depth parity (first 4 of 40 real layers plus embeddings and the final norm): KL(HF‖TL) 6.5e-9 to 5.3e-7, max log-prob deviation 3.8e-6 to 7.6e-6, argmax agreement 4/4 on the same four prompts the other models' gates use.
  • Pruning score extraction (build_importance_dict) and quantization target resolution — both resolve real modules correctly on the full checkpoint (module-path resolution with synthetic scores; no pruned or quantized model is evaluated) (tests/regression/test_aya_expanse_32b_interventions.py).

What does not run yet, and why:

  1. The model does not fit on most single GPUs. The weights are about 60 GiB at fp16/bf16, and TransformerLens keeps a separate W_U (~3.9 GiB more), so a loaded TransformerLens copy needs about 64 GiB for weights alone. Full-depth parity, discovery, evaluation, and weight steering therefore need multi-GPU loading through n_devices.
  2. n_devices loading is broken in transformer-lens==3.8.0. HookedTransformer.move_model_modules_to_device places each block with get_best_available_device(cfg), which picks the device with the most free memory and ignores the block index. The forward pass moves the residual stream per block with the index-based get_device_for_block_index(i, cfg). The two disagree. Symptoms: RuntimeError: Expected all tensors to be on the same device inside TransformerLens's layer_norm.py, or nearly the whole model placed on one GPU followed by an out-of-memory error.

This is not a defect in the shared cohere1 converter. Do not treat 32B as having the discovery, evaluation, or weight-steering coverage the other four models on this page have.

SmolLM3-3B (CIRCUITKIT_RUN_SMOLLM3=1, public repo, no HF_TOKEN needed):

CIRCUITKIT_RUN_SMOLLM3=1 python -m pytest \
    tests/backends/tl_compat/test_smollm3_parity.py \
    tests/regression/test_smollm3_discovery.py \
    tests/regression/test_smollm3_evaluation.py \
    tests/regression/test_smollm3_interventions.py -v

# ACDC only:
CIRCUITKIT_RUN_SMOLLM3=1 CIRCUITKIT_RUN_SMOLLM3_ACDC=1 \
    python -m pytest tests/regression/test_smollm3_discovery.py -v -k acdc

(Tiny Aya's own gates use CIRCUITKIT_RUN_TINY_AYA=1 — see Tiny Aya's Tests section.)

What each gate proves:

  • Parity (test_*_parity.py) — TL-vs-HF next-token KL divergence < 1e-4 on short prompts (and past the 4096-token sliding-window boundary for Command R7B). This is the only test that numerically catches a wrong RoPE interleaving choice, LayerNorm fold, embedding tie, or logit_scale fold. Observed on real weights: Aya Expanse KL ≈ 1e-7–4e-7 (four orders of magnitude under the gate); SmolLM3-3B KL ≈ 2.9e-8–1.1e-7 (four to five orders of magnitude under the gate), with argmax agreement and the exact per-layer NoPE mask ({3, 7, 11, ..., 35} skip rotary) both confirmed directly on the loaded model. These figures are from manual real-weight GPU runs; CI does not reproduce this parity measurement. Aya Expanse 32B's parity gate is a truncated-depth variant of this same check (first 4 of 40 real layers plus embeddings and the final norm, built by reading only the safetensors shards those tensors live in): KL(HF‖TL) 6.5e-9 to 5.3e-7, max log-prob deviation 3.8e-6 to 7.6e-6, argmax agreement 4/4. See the warning above for what this covers and why the full-depth variant is blocked.
  • Discovery (test_*_discovery.py) — end-to-end discover_circuit on greater_than for 5 of the 6 stable algorithms (eap, eap-ig, eap-gp, ibcircuit, cdt); asserts finite, non-degenerate node scores.
  • Evaluation (test_*_evaluation.py) — circuitkit.api.evaluate_circuit faithfulness (patching + ablation pillars) on a cdt-discovered greater_than circuit, loading the model once and passing the same handle to both discovery and evaluation (avoiding a second 7–8B load). evaluate_circuit unconditionally sets the same qkv-flag activations (use_attn_result/use_split_qkv_input/use_hook_mlp_in) that only the EAP-family algorithms need during discovery, regardless of which algorithm produced the circuit being evaluated — at smoke-test batch/example counts this ran cleanly on GPU for all four models, but a future maintainer raising those counts on a 7–8B model may need the same CUDA_VISIBLE_DEVICES="" CPU fallback discovery needed.
  • Interventions (test_*_interventions.py) — pruning score extraction, quantization target-module resolution, and weight-steering setup, all on real weights for Command R7B/8B/SmolLM3-3B. For Aya Expanse 32B, pruning and quantization resolution also pass on the real checkpoint; weight steering is blocked by the same confirmed transformer-lens multi-GPU bug described in the 32B warning above. See Architecture Registry for exactly what was and wasn't run per model.

ACDC is excluded from the standard gate

acdc (the sixth stable algorithm) needs the same per-head qkv-flag activation blow-up as the EAP family. CircuitKIT's own OOM preflight correctly refuses this on a single 47 GB GPU for a 7–8B model; the only fallback is CPU, where a single ACDC run — already scoped to one tao value instead of the default 8-combination grid — took over 2.5 hours on a 7B model without finishing. That is not a practical regression gate, so each model's ACDC test is kept (it does work — verified to launch, wire up ioi, and run) but skipped by default behind a second, separate opt-in (CIRCUITKIT_RUN_<FAMILY>_ACDC=1) so it doesn't silently cost hours on every real-weight run. This exclusion is standing policy for future large-model discovery regression suites, not specific to these three models.

GPU memory reality

None of CircuitKIT's discovery algorithms currently shard activations across multiple GPUs. For a 7–8B model, the qkv-flag algorithms (eap, eap-ig, eap-gp, acdc) and ibcircuit's own fixed-batch training step both exceed a single 47 GB GPU's headroom and need to run on CPU (CUDA_VISIBLE_DEVICES="") — slower (tens of minutes), but correct, and the repo's own OOM preflight refuses the GPU attempt loudly rather than crashing mid-forward. cdt needs neither. SmolLM3-3B (~6 GB bf16) is small enough that its entire discovery gate ran on GPU with no CPU fallback.

Aya Expanse 32B needs more memory than the rest of this page: about 60 GiB of weights at fp16/bf16 (14 safetensors shards), and TransformerLens keeps a separate W_U (~3.9 GiB more), so a loaded TransformerLens copy needs about 64 GiB for weights alone. That does not fit on most single GPUs, and a full fp32 copy needs about 129 GiB. Multi-GPU loading through n_devices fails on transformer-lens==3.8.0 however much GPU memory is free (see the warning in Tests). The parity gate avoids the full load by never materializing more than 4 real layers through TransformerLens. Pruning and quantization resolution load the HF copy directly with device_map="auto", which the TransformerLens bug does not affect. Full-depth parity, discovery, evaluation, and weight steering need the n_devices path and do not run yet.


See also

  • Tiny Aya (Cohere2) — the original cohere2 port these three models extend; its Tests section documents CIRCUITKIT_RUN_TINY_AYA.
  • Architecture Registry — the cohere family (now covering Tiny Aya, Command R7B, and Aya Expanse) and the new smollm3 family, plus what the intervention validation actually covered per model.
  • circuitkit.backends._tl_compat — the port source (registry.py, cohere.py, smollm3.py).