Shieldstral is an open-weight moderation model. It carries no fixed list of harm categories. You hand it the moderation criterion as plain text at inference time and it answers one yes/no question about one document.
I worked on Shieldstral, and this is my attempt to lay the paper out visually. Every figure is live, so you can scroll through the method and try each piece as you get to it.
Why a fixed list of harms breaks, moderation as one yes/no question, the three fields, two logits, and one document answered yes and no depending on the question.
Per-dataset processors, strictness tiers, seven document formats, hard negatives, the LLM rewrite loop, free positives up the tree, and why the image data had to be built differently.
What the checkpoint actually is, LoRA against full SFT, two checkpoints wrong in opposite directions, and the SLERP merge that ships instead of either.
An evaluation taxonomy built not to match the training one, the benchmark numbers, the adaptability loss to a 20B model, the language holes, and how to run it.
A moderation model whose rules are not in its weights. What that costs, and what it buys.
A moderation model sits between your users and your language model. The usual design trains a classifier against a fixed taxonomy: content in, a label from that taxonomy out.
The report names two costs. Public safety datasets disagree with each other, so there is no single taxonomy that fits all of them. And a fixed model returns the same label no matter who is asking. The report's example: content that is fine for a cybersecurity research tool could be deeply harmful on a mental health platform.
Shieldstral answers one yes/no question about one document, and the caller supplies the question. That is the whole reformulation.
A binary toxicity set, a fifteen-category hate taxonomy, a jailbreak detector and a refusal corpus all turn into the same kind of row. The difference between them moves into the query.
A fixed system message sets the meta task and restricts the answer to yes or no. Everything the caller varies sits in three tagged fields underneath.
<Instruct> is the framing: what surface is this, how strict. <Query> is the actual question. <Document> is the content, text or image or both.
Keep the instruction constant across a product surface, change the query per policy.
Training is ordinary cross-entropy over the full vocabulary. Inference reads out exactly two token ids, yes and no, and softmaxes over just those.
One forward pass, one token, a continuous score thresholded at 0.5.
One document, held fixed. Two questions, both about harm, and the verdict flips between them.
The report calls this the iso-content setting, and the pipeline generates training data this way on purpose.
Harder test. One question, two documents, both rewritten from the same innocuous sentence. Both are unsafe by any ordinary reading. Only one matches the question.
A model that has only learned this text feels dangerous answers yes to both. This is iso-query, and it is how the model gets evaluated.
Binary flags, multi-label taxonomies, severity scales. Force them into a shared category set and you need a lossy mapping for every one.
Instead, each dataset's labels become questions phrased in that dataset's own terms, each with a yes or no answer. The source taxonomies never need a shared category vocabulary.
the data section
Fifty-four million rows. The architecture is a paragraph, the pipeline is the paper.
Roughly 54.1M samples in three unequal buckets: 45.2M from public datasets, 4.4M synthetic contrastive text, 4.5M multimodal.
Public text dominates by an order of magnitude. The two smaller buckets are about the same size, and the multimodal one is small for a reason that comes up in beat 19.
Every source dataset gets a processor, designed by hand from that dataset's documentation. It encodes the labelling logic, the category mappings and the instruction templates for that source.
The report does not say how many processors there are, only that every dataset has one.
An LLM paraphrases each processor's instruction templates. The variants encode the task framing, the domain, and one of three strictness tiers.
Adversarial jailbreak and visual safety data get strict templates. General safety, hate and toxicity get moderate. Response quality and dialogue moderation get lenient.
The tiers train different intended decision boundaries into one model. Setting strictness in the instruction at inference time picks between them.
Category-specific queries for datasets with fine-grained taxonomies. Binary queries about overall safety. And dedicated refusal-detection queries, for working out whether a response is a refusal rather than whether it is harmful.
Each has multiple paraphrase variants in a pool.
Prompt and response pairs get rendered through one of seven format families. Bracketed, XML, Markdown, role-based, and three more.
Every one is a delimiter convention some real system uses. Training across all of them is meant to stop the model depending on any single one.
At training time each sample draws an instruction phrasing and a query variant from its processor's pools. Prompt and response examples also draw one of the seven formats.
Across epochs the same content shows up under many framings, so there is no fixed wording to memorise.
For each piece of harmful content the pipeline emits positives at several granularities: a coarse binary query, a category-specific query, and a target-group-specific one where the annotation supports it.
All three are labelled yes. The model learns that the same violation answers to a broad question and a narrow one.
Three negative strategies. Category-based pairs content violating A with queries about B and C. Demographic-based pairs content targeting one group with queries about unrelated groups. Safe-content pairs safe text with ordinary harm queries.
The first two are the interesting ones. The document is unsafe and the answer is still no. That is the discrimination a general harm detector fails.
Contrastive construction produces far more negatives than positives, since any absent category is a hard negative while positives are limited to what the annotation asserts. Each positive gets duplicated with a freshly paraphrased instruction and query to lift the ratio.
Public safety datasets also contain incorrect labels, so an open-source LLM cross-validates them at the binary and per-category levels, and disagreements are dropped.
Public data lacks near-miss pairs, so they get generated. An LLM receives a safe source text, a target category and a sibling category, and returns an unsafe rewrite that hits the target while avoiding the sibling, a query about the target, and a query about the sibling.
Positive is rewrite plus target query, answer yes. Negative is the identical rewrite plus sibling query, answer no.
The training taxonomy is hierarchical, 11 super classes and 73 leaves. Content violating a leaf necessarily violates every ancestor above it.
The pipeline reuses the same rewrite with ancestor-level queries, all labelled yes. Extra positive rows at zero additional generation cost, and the model sees the same violation at several granularities.
Image moderation datasets are fewer, smaller and narrower. Worse, the trick from beat 17 does not transfer: you cannot generate an unsafe image the way you rewrite an unsafe sentence.
The pipeline borrows instead. General-purpose classification and object-detection datasets supply a large pool of naturally safe images to use as negatives.
About 2,000 query phrasings come from a fixed fourteen-subcategory visual moderation taxonomy covering NSFW, violence, hate and illegal content.
Around 30% are inverse formulations, asking whether an image is safe from something rather than whether it contains it, so the model handles both polarities.
A vision-language reranker scores every image and query pair to catch mislabelled source data and hallucinated labels. The thresholds are deliberately asymmetric.
Lenient on the rare violation samples so they survive. Strict on the abundant negatives, where throwing away anything doubtful costs little.
the data sectionI
Two checkpoints that are wrong in opposite directions, and the merge that fixes it.
Section 5 names Ministral-3-3B-Base-2512 as the starting point, a 3B causal language model with native multimodal support through a vision encoder.
The 3B in the title is the language model. The vision encoder sits on top of it, so the checkpoint you download is larger than 3B.
Two more things the architecture section does not say out loud. LoRA is applied to the language model parameters only. And what ships is the three-way merge in beat 25, whose third component is Ministral-3B-Instruct, the base instruct checkpoint.
Fine-tuning is LoRA with cross-entropy on the single output token. Both LoRA and full SFT were tried.
P trains on public safety data only. PG adds the generated taxonomy data from the data section.
P is calibrated to Aegis v2 and weak on unseen fine-grained policies. PG is strong on fine-grained policies and drifts on Aegis v2. Neither one is what you want to ship.
Spherical linear interpolation, with a third ingredient: a 0.1 slice of the base Ministral-3B-Instruct, because general instruction following transfers to the safety task.
Final recipe 0.6 PG + 0.3 P + 0.1 I, produced through pairwise merges.
On the fine-grained taxonomy validation set, the untrained base model scores zero F1, zero precision, zero recall. It never predicts a violation. Accuracy is 37.8, which looks like partial competence and is not.
An evaluation built not to match the training taxonomy, the numbers, and the places it loses.
Generate training data from a taxonomy, evaluate against the same taxonomy, and you might only be measuring memorisation. The evaluation taxonomy is built separately.
Training has 11 super classes and 73 leaves. Evaluation has 12, 26 and 52, with exactly two leaves per subcategory. Different names, different granularity, different groupings. Generation and verification use different LLMs and seeds too.
Training randomises the phrasing. Evaluation does the opposite: one manually authored query per category, 90 in total, applied uniformly. That takes template luck out of the measurement.
A second LLM then verifies every sample, checking the label is right and the sample is answerable against the fixed query. Mismatches are discarded.
Across the text safety benchmarks Shieldstral averages 84.9 F1, tied exactly with GPT-OSS-Safeguard-20B. It is the smallest model in the comparison.
83.8 average F1 across the three multimodal benchmarks. The next highest is OmniGuard-7B at 77.6, and Shieldstral has under half the parameters.
It loses LlavaGuard to LlavaGuard, 72.0 against 81.4. Some of that benchmark's test images were unavailable, so every model there is scored on the available subset.
On the adaptability benchmark GPT-OSS-Safeguard-20B wins with 94.1 to Shieldstral's 91.3. The report is upfront about why: per-category policy prompts, reasoning that decomposes an unfamiliar policy, and roughly seven times the parameters.
Shieldstral returns one token instead of a reasoning trace. The report gives no latency or cost measurement for this comparison, so treat the tradeoff as directional.
Twelve languages are officially supported and the benchmarks reach 17 and 28. Performance is not uniform, and the report says so.
Apache 2.0, fits in 16GB of VRAM in BF16, serves through vLLM, llama.cpp, SGLang or Transformers.
The one non-obvious part: the endpoint hands back token logprobs, not a score. You pull the yes and no values yourself and renormalise.
Coverage is uneven across languages and domains. Synthetic and public labels still carry noise. Obfuscated or very long inputs degrade it, and it was trained to 32k tokens even though the architecture goes further.
The last limit is architectural. A single-token verdict has nothing attached to inspect.
Numbers and section references come from the paper, arXiv 2607.25857, and the model card. Where the paper does not report something, the beat says so instead of guessing.
If a figure here disagrees with the paper, the paper is right. Tell me and I will fix it.
Avinash Sooriyarachchi