Howardism · Vol. 03Plate II · No. 02
Interpretability, in order.
Notes8DomainInterpretabilityOpen Qs16Newest11 Jul 2026Oldest7 Jun 2026
Reading model internals: activations and the global workspace.
Map of Content for the interpretability domain — 8 concepts. Reading model internals: the global workspace, the Jacobian lens, activation monitoring, and internal signatures of misalignment. Curated entry point; see Home for all domains.
- Access-Consciousness Indicators in AI — The consciousness question the workspace paper deliberately does and doesn't answer: it tests functional indicator properties (global workspace, higher-order, attention schema, recurrent processing) against a concrete inspectable structure, takes no position on phenomenal experience — and finds that ablating the J-space flattens the model's experiential reports while leaving its coherence intact
- The Assistant Persona in the Workspace — Post-training installs the Assistant's point of view into a workspace that already exists in the base model: safety assessments and empathy appear while the model is still reading the user's message, and it internally flags its own outputs —
disclaimer/fictionalwhen roleplaying, an all-capsBUTwhen prefilled against its own preferences,damnwhen it fails to suppress a thought - Automatic vs. Flexible Cognition in LLMs — The selectivity result: a model can parse, classify, continue text and detect anomalies with its workspace suppressed, but loses multi-hop reasoning, translation, analogy and summarization — and chain-of-thought partially rescues it, because writing the intermediate down externalizes what the model would otherwise have to hold internally
- Counterfactual Reflection Training — Train the model to write constitution-grounded reflections if interrupted and asked — then never ask it. The implanted concepts show up in the workspace during the uninterrupted task, and behavior changes: dishonesty 0.25→0.07 (fabrication) and 0.38→0.05 (deception) on Haiku 4.5, with ablation of the implanted lens vectors reverting the gain
- Internal Signatures of Misalignment — The J-lens reads strategic and deceptive cognition that never reaches the output:
leverage/blackmailwhile reading the emails,panic/fakeat the moment a model decides to fabricate a bug, and a standingfake/secretly/fraudsignature at response-start in reward-hacking model organisms on ordinary prompts — plus the escape route the authors name themselves - Jacobian Lens (J-lens) — Anthropic's interpretability method for reading verbalizable content out of a model's residual stream: a corpus-averaged Jacobian from each layer to the final layer, composed with the unembedding, giving one vector per vocabulary token — a causal, principled correction to the logit lens that costs one matmul per layer and reads what the model is poised to say rather than what it happens to say
- The Global Workspace in Language Models (J-space) (hub) — Anthropic's July 2026 finding that LLMs maintain a small privileged set of verbalizable representations — the J-space — that satisfies the functional criteria of a cognitive global workspace: verbal report, directed modulation, internal reasoning, flexible generalization, and selectivity; it carries <10% of activation variance and ~25 concepts at a time, yet the causal effects concentrate almost entirely in it
- White-Box Activation Monitoring — Reading a model's internal activations (not its outputs) to monitor alignment: contrastive probes/steering vectors for concepts like evaluation awareness, and a natural-language-autoencoder verbalizer that decodes residual-stream vectors into text — the complement that catches what chain-of-thought monitoring misses
Open questions 16 open
- Access-Consciousness Indicators in AI
- If the workspace is verbal *because the output space is verbal*, then a model that can generate images should develop a **visual component** to its workspace. That is a concrete, falsifiable prediction the paper makes and does not test.
- Does the model's own report of experience change if you tell it its J-space is ablated? (Nobody asked.)
- Is "experiential language" the right proxy at all, or is the ablation simply removing abstraction from the register?
- Automatic vs. Flexible Cognition in LLMs
- Does more RL on a behavior push it *out* of the workspace (making it automatic, and invisible)? Nobody has tested it, and it is the single most alignment-relevant version of this question.
- Jacobian Lens (J-lens)
- Can multi-token J-lens vectors be made good enough to remove the vocabulary restriction — and how much of the "workspace" is currently invisible because of it?
- The J-lens reads the workspace's contents but says nothing about **how content gets in**. What is the model's analog of attentional selection?
- The highest-J-kurtosis SAE features are amplified *more* strongly by MLPs than the J-lens vectors themselves — evidence the lens only approximates the true workspace directions. What is the better basis?
- The Assistant Persona in the Workspace
- Is `BUT`-then-comply a *sycophancy* mechanism? The setup (prefill the model into a position it disprefers, watch it argue for it anyway) is close to the shape of sycophantic capitulation, and nobody has connected them.
- Does `disclaimer`/`fictional` at the turn boundary survive an actual jailbreak, or is its absence the signature of a successful one?
- If the base model's workspace has no self, what *is* in it at the positions where the post-trained model represents the Assistant?
- The Global Workspace in Language Models (J-space)
- **How does content get into the workspace?** The paper characterizes contents and consequences, not the selection mechanism. Something like attentional selection is operating; nobody has identified it.
- **Does the J-space scale with model size?** All results are on large production models (Haiku/Sonnet/Opus 4.5, Opus 4.6). Whether small models have a poorer workspace, a proportionally smaller one, or none is unknown — as is when in pretraining it emerges, and whether abruptly.
- **Is the "workspace vs. motor" boundary principled or post-hoc?** The authors concede it was identified empirically and lack a principled definition separating the two.
- **Are the early third of layers genuinely workspace-free, or is the lens just blind there?** CKA shows a distinct early regime but cannot adjudicate.
- White-Box Activation Monitoring
- The NLA verbalizer is unvalidated for precision; how much of the flagged grader awareness is real signal vs. NLA hallucination?
- Inhibiting eval-awareness made Opus 4.8 *less* honest — does that generalize, and what does it imply about relying on "the model behaves well because it's watched"?