Governance & Workforce, in order.

Open questions 42 open

• AGI-to-ASI Pathways
  • For each friction: is it a *fundamental blocker* (multi-year plateau) or a mere *friction* (slows, doesn't halt)? The report's central unresolved question. **Synthesized with Anthropic:** [[wiki/derived/rsi-growth-curves-which-friction-binds]] — data-wall and research-gets-harder demote themselves into compute; economics and neural-paradigm are pathway-conditional; the abstraction barrier is the candidate fundamental (re-pacing) blocker; and deliberate slowdown is the only *exogenous* friction — the one Anthropic wants to install and this report doubts can be made to bind.
  • Do the four pathways compound multiplicatively when run in parallel, and how would we detect that early?
  • Can benchmarking methodology that doesn't saturate at human level be built before it's needed for ASI?
• AI Accelerating AI Development
  • LOC, self-reports, and headroom-dependent multiples all overstate; what *unbiased* throughput metric would Anthropic's promised shift to "direct measurement of AI R&D acceleration and researcher uplift" ([[ai-rd-autonomy-evaluation]]) actually use?
  • The W2S result didn't transfer to production-scale models. Is that a temporary scaling artifact or a structural limit on autonomous research?
  • The next-step judgment trend (51%→64%) is measured only on weak-human-move slices. What does the curve look like on a representative sample of research decisions?
• AI R&D Autonomy Evaluation (AECI)
  • "Not close to substituting for senior researchers" is a subjective, internally-sourced judgment. What objective signal would replace it as models approach the threshold?
  • AECI is a single scalar fork of an external index; how sensitive is the 155.5 / frontier-not-advanced conclusion to the choice of the n=11 evaluation set?
  • The shift to "direct measurement of AI R&D acceleration and researcher uplift" is announced but not yet operationalized in this card — what does that measurement look like?
• Autonomous Scientific Discovery
  • Every result is Anthropic-reported and example-selected; the genomics "100× smaller beats *Science*" claim is "intend to publish" — what survives external peer review?
  • Science's verification gap: the formal-proof loop self-validates; here a wrong-but-confident hypothesis costs a wet-lab cycle to falsify. Does autonomy without a fast verifier *increase* the verification bottleneck rather than relieve it?
  • If hypothesis-generation is genuinely at ~80% preference, how much of "research taste" is left as a distinctively human function — and how would you measure the residue?
• Capability-Gated Model Fallback
  • The >95%/<5% figures are session-level; what's the false-positive rate for *legitimate* security researchers and biologists, whose benign queries are exactly the ones most likely to trip the conservative classifiers?
  • Fallback-not-refusal preserves UX but means the *real* general-access model for security/bio-adjacent work is Opus 4.8, not Fable — does that quietly cap Fable's value for whole professional segments until the trusted-access programs open?
  • The UK AISI's "progress toward a universal jailbreak" is disclosed but not quantified — and the post-launch **access suspension** (see [[claude-fable-5]]) raises the question of whether a safeguard failure forced it.
  • Does swapping to a weaker model on flagged topics create an exploitable oracle (probe which queries trigger fallback to map the classifier's boundary)?
• Effective Compute Scaling
  • When does more compute reliably yield more *intelligence* — only for some problem classes, or generally? Can quantitative and qualitative scaling be traded off?
  • Can data generation (synthetic, simulated, interactive) actually keep pace with model-size growth, or does the data wall bind first?
  • When (if ever) does scaling become economically unviable, and how do hardware/software-efficiency trends move that point?
• Frontier Pause Verification
  • What does an AI-training "verification regime" concretely consist of — compute-accounting, datacenter inspection, hardware attestation, on-chip telemetry? The essay names the problem, not the mechanism.
  • Detectability < verifiability: can detection even be made reliable when training runs leave no physical signature and inputs are dual-use?
  • Who adjudicates triggers and lifts? No institution currently holds that mandate, and standing one up is itself a decade-scale task.
• Intelligence Explosion Dynamics
  • Can "recursive improvement scaling laws" be formulated — predicting self-improvement curves (and their plateau point) from early-onset datapoints?
  • How far can a *fixed* model's performance be pushed with test-time search alone, and under what conditions does recursive distillation degenerate vs. compound?
  • Which binds first — algorithmic ceilings, the embodied bottleneck, or compute/energy supply — determining exponential vs. hyperbolic vs. S-curve? **Synthesized:** [[wiki/derived/rsi-growth-curves-which-friction-binds]] — both this report and Anthropic's locate the binding constraint *outside* cognition (the slowest un-acceleratable step coupling the loop to reality); the embodied bottleneck re-paces rather than halts, data-wall/research-harder demote into compute, and the abstraction barrier is the one candidate *fundamental* blocker.
• Multi-Agent Collective Intelligence
  • Do homogeneous LLM collectives produce real synergy, or only humans-with-human-limits benefit from division of labor?
  • What's the actual shape of "multi-agent scaling laws," and does it depend on organization form (homogeneous collective vs. heterogeneous market) or task complexity?
  • Is running more instances more compute-efficient than making individual models larger (up to a single monolithic system)?
  • How do humans meaningfully interact with and steer very large agent groups operating at superhuman speed and output volume?
• Recursive Self-Improvement
  • Is "research taste" a true ceiling (future 1) or just the next capability to fall (futures 2–3)? The essay frames this as the single load-bearing uncertainty.
  • The RSI extrapolation rests on trends staying exponential rather than S-curving — but the essay concedes it cannot rule out an architectural ceiling or a compute/energy supply-chain constraint. Which binds first? **Synthesized against DeepMind:** [[wiki/derived/rsi-growth-curves-which-friction-binds]] — the three futures map one-to-one onto DeepMind's three growth shapes; the first friction to bind is the already-binding one (Amdahl's-law verification/oversight = DeepMind's embodied bottleneck), and the abstraction barrier supplies the mechanism Anthropic lacks for whether taste is a *real* ceiling (Future 1).
  • If misalignment compounds through self-improvement (future 3), is AECI-gated [[responsible-scaling-policy-evals|RSP]] review fast enough to catch it before control is lost?
• Research Taste as the Human Bottleneck
  • Is research taste a genuine ceiling (an architectural capability scaling can't reach) or the next jagged valley to fill? The essay calls this the decisive unknown.
  • If taste is automatable, what — if anything — remains a durable human comparative advantage in AI development?
  • How do you measure rubber-stamping? "Humans set direction" can be true on paper while real judgment quietly transfers to the model.
• Responsible Scaling Policy Evaluations
  • The RSP determination leans heavily on "we use it daily and it doesn't substitute for our researchers." How well does that subjective judgment scale as models approach the threshold?
  • The two new general-access risk pathways (other AI developers; major governments) are newly in scope but lightly evaluated — what would a positive finding there even look like?
  • How does the RSP brake interact with [[recursive-self-improvement]]: is AECI-based gating fast enough if acceleration compounds, and does single-lab gating even matter without the multilateral [[frontier-pause-verification|pause-verification]] regime?
• Returns to Expertise in Agentic Coding
  • The forward test the report itself names: do the returns to expertise **persist, narrow, or invert** as models improve? A decrease would mean models are absorbing the judgment users currently supply.
  • Outcomes are transcript-inferred (verified success leans on git activity + explicit affirmation). How much of the management edge — and the whole success gradient — is *real* outcome vs. who-narrates-success-in-the-transcript?
  • The study excludes headless / SDK / IDE usage (a "substantial share"). Does the returns-to-expertise pattern hold in non-interactive and pipeline use, where there is no human steering mid-session at all?
  • Is "intermediate captures most of the benefit" stable, or an artifact of current model capability — i.e., will the concave curve flatten further (everyone converges) or steepen (mastery starts to separate again) as models get better?