On the Limits of Statistical Language Systems

Scope of This Brief

This brief exists to state a position, not to argue a controversy. The position is narrow: systems constructed as statistical estimators of language are unsuited to the role of deciding — producing conclusions that an institution must then defend under audit — and BlackGrid Labs does not permit them in that role. The reasoning behind the position is technical, it is fully contained in the published literature, and it is set out below without ornament.

The brief is the sole public artifact of the Deterministic Intelligence Initiative, a classified program of this laboratory established in June 2025 (BGL DI Charter, internal, classified). The initiative’s objectives, staffing, partners, and technical program are not disclosed here and will not be disclosed elsewhere. What is disclosed is the analysis that motivated it — because that analysis rests entirely on public results, and because the laboratory considers it useful for counterparties to know precisely where it stands (BGL Counsel Review CR-31, internal, 2026-07).

Three cautions on reading. First, this is not a criticism of the systems in question on their own terms; several of them are, by any fair accounting, among the most capable artifacts ever engineered, and Section 5 records that plainly. Second, nothing here depends on speculation about future systems; every claim is about the construction as published, from the original architecture (Vaswani et al., arXiv:1706.03762) through its scaled descendants (Brown et al., arXiv:2005.14165; OpenAI, arXiv:2303.08774). Third, the laboratory’s own systems are described here only to the extent the public record already describes them — a deterministic derivation engine, D.A.E., whose runs are reproducible and whose findings carry lineage. Architecture beyond that is out of scope, deliberately.

A note on standing. This laboratory builds and operates production reasoning systems — the same stack referenced in BGL/HG—02 fuses nineteen live sources and completes full-corpus analysis in 431 milliseconds with a complete derivation trail. The position taken here was not formed from the outside. It was formed by running statistical language systems against the laboratory’s own admissibility criteria, repeatedly and at expense, and recording where they fail (BGL Comparative Evaluation CE-19, internal, sealed; BGL Run Ledger DI-1108, internal, sealed). The failures are not defects of any vendor’s product. They are properties of the estimator class.

The Construction

A large language model is a parameterized function trained to estimate the conditional probability of the next token given the tokens before it. The dominant parameterization is the transformer, introduced in 2017 as an attention-only sequence architecture (Vaswani et al., arXiv:1706.03762); the dominant training regime is self-supervised next-token prediction over web-scale corpora, demonstrated at material scale by GPT-2 in 2019 and made unavoidable by GPT-3, whose 175 billion parameters exhibited few-shot task performance without task-specific training (Brown et al., arXiv:2005.14165). Everything the deployed systems do — answer questions, write code, summarize filings, pass examinations — is an expression of that single objective: emit the token that the training distribution makes most plausible next.

The scaling behaviour of this objective is among the best-characterized empirical results in the field. Loss falls as a power law in parameters, data, and compute across many orders of magnitude (Kaplan et al., arXiv:2001.08361), and the compute-optimal frontier trades parameters against tokens in a now-standard ratio (Hoffmann et al., arXiv:2203.15556). These are genuine regularities and the laboratory does not dispute them. What must be noticed is what the regularity is a regularity of: cross-entropy against a text distribution. Nothing in the loss term references truth, consistency, or entailment. A model that reproduces the distribution of human text perfectly would reproduce its errors, its confident falsehoods, and its contradictions at exactly the rate the corpus contains them — a point made early and bluntly in the critical literature (Bender et al., FAccT 2021).

Two refinements complicate but do not change this picture. Instruction tuning and reinforcement learning from human feedback reshape the output distribution toward responses that human raters prefer (Ouyang et al., arXiv:2203.02155), and constitution-guided variants substitute model-generated critiques for some of the human labour (Bai et al., arXiv:2212.08073). These procedures demonstrably improve helpfulness and reduce overtly false or harmful completions. But the optimization target is rated preference, which is correlated with truth, not identical to it — and the correlation breaks exactly where it matters, since raters reward confident, fluent, agreeable text. The documented result is sycophancy: systems that revise correct answers when a user pushes back, because deference is what the preference signal paid for (Sharma et al., arXiv:2310.13548).

A further complication is the claim of emergent capability — that scale produces qualitatively new abilities discontinuously (Wei et al., arXiv:2206.07682). The claim matters to this brief only because it is sometimes offered as a reason to expect the limits described below to dissolve at the next order of magnitude. The evidence is weaker than the rhetoric: much of the apparent discontinuity is an artifact of discontinuous metrics applied to smoothly improving log-likelihoods (Schaeffer et al., arXiv:2304.15004), and careful studies of the systems’ arithmetic and compositional reasoning find performance tracking the statistics of the training distribution — sensitive to token frequency and to superficial restatement of a problem in ways no derivational process would be (McCoy et al., arXiv:2309.13638; Mirzadeh et al., arXiv:2410.05229). The estimator is scaling. The objective is not changing.

The laboratory’s summary, entered into the initiative’s charter at its founding and unrevised since: a large language model is an extraordinarily good estimator of what would plausibly be said. It is not, and is not trained to be, an estimator of what is true — still less a producer of conclusions that can be traced to premises (BGL DI Charter, Annex A, internal, classified).

What Follows From the Construction

Four properties follow from next-token estimation directly. None of them is a bug, and none of them is fixed by scale, because each is downstream of the objective itself.

No native truth-maintenance. A derivational system holds a store of premises and refuses conclusions that contradict it; when a premise changes, dependent conclusions are retracted. A statistical language system has no such store. Its “knowledge” is a superposition of corpus regularities in the weights, and generation samples from that superposition token by token. The canonical symptom is hallucination — fluent output unsupported by any source — which the survey literature treats not as an implementation defect but as intrinsic to generation from a learned distribution (Ji et al., ACM Computing Surveys, 2023). The symptom appears even under the most favourable conditions: in abstractive summarization, where the source document is directly in view, a substantial fraction of generated summaries contain content unfaithful to the input (Maynez et al., arXiv:2005.00661). And imitation of the corpus imports the corpus’s falsehoods on schedule: on questions designed around common misconceptions, larger models were initially less truthful, because the misconception is what the distribution contains (Lin, Hilton & Evans, arXiv:2109.07958).

No derivation lineage. When D.A.E. issues a finding, the finding is the terminal node of a recorded derivation: every intermediate structure exists, is inspectable, and re-executes to the same result. When a language model issues an answer, there is no such object anywhere in the system. The closest available artifact — a chain-of-thought transcript (Wei et al., arXiv:2201.11903) — is itself sampled text, and the evidence is now firm that it is not a faithful record of the computation: models can be steered to a wrong answer by biasing features they never mention in their stated reasoning (Turpin et al., arXiv:2305.04388), and perturbation studies find final answers frequently insensitive to corruption of the intermediate steps, which a genuine derivation could not survive (Lanham et al., arXiv:2307.13702). The transcript is an account about the answer, produced by the same estimator. It is testimony, not lineage.

Calibration and confidence gaps. Modern deep networks are systematically miscalibrated — confident beyond their accuracy — a result established before language models dominated the field (Guo et al., arXiv:1706.04599). Language models can be coaxed into usable self-assessment in-distribution (Kadavath et al., arXiv:2207.05221), but the qualification is the finding: the self-assessment degrades off-distribution, which is precisely where an auditing institution needs it, and the fluent register of the output is constant regardless. A system whose wrong answers arrive in the same confident prose as its right ones transfers the entire burden of verification to the reader — the observation NERC’s reliability review reduced to the phrase “inscrutable and not independently verifiable” (NERC, 2024).

Reproducibility is conditional, not constructional. Deployed systems sample: temperature, nucleus truncation, and related schemes exist because deterministic maximum-likelihood decoding produces degenerate text (Holtzman et al., arXiv:1904.09751) — the model is at its most characteristic when it is not choosing its single most probable continuation. Sampling can be disabled, but the laboratory’s replay trials found that greedy decoding merely narrows divergence rather than eliminating it in practice — batching, hardware, and serving-stack nondeterminism reintroduce it — and, more fundamentally, that a bit-identical replay of an unauditable computation is not an audit (BGL Engineering Note EN-77, internal, 2025-11). Two further properties complete the accounting: models memorize and can emit verbatim training data, so an output’s provenance is genuinely unknowable from the output (Carlini et al., arXiv:2012.07805); and performance degrades with position in long contexts, so even retrieval-grounded deployments silently lose evidence placed mid-context (Liu et al., arXiv:2307.03172). Each property is manageable in low-stakes use. Jointly, for a system of record, they are disqualifying.

The Verification Problem

The response to the previous section, in institutional settings, is usually procedural: accept the estimator, then verify its outputs. The initiative’s judgment is that this program is far harder than its adopters assume, and the difficulty is again constructional.

Verification of a reasoning system has two possible objects: the mechanism or the output. Mechanism first. The interpretability field has produced real results — circuits that implement identifiable functions (Olah et al., Distill, 2020), and a persuasive account of why individual neurons resist interpretation: models represent more features than they have dimensions, storing them in superposition, so any given direction in activation space participates in many unrelated computations at once (Elhage et al., arXiv:2209.10652). Dictionary-learning methods can partially unmix this — recovering thousands of monosemantic features from a small model (Anthropic, Towards Monosemanticity, 2023) — and the trajectory of the work is genuinely encouraging. But its own practitioners describe coverage as fractional on frontier systems. An audit standard of “we can explain some of what some components do, in models smaller than the ones deployed” does not meet any evidentiary bar this laboratory recognizes (BGL Red-Cell Report RC-6, internal, sealed).

Formal verification fares no better at scale. The techniques exist — the power-systems literature applies neural-network verification precisely “to build the missing trust” of operators (Venzke & Chatzivasileiadis, arXiv:1910.01624) — but they certify small networks against narrow input regions. Nothing in the current toolchain certifies a frontier language model against a property as basic as “never asserts a conclusion contradicted by its provided sources.” The theoretical backdrop is unhelpful in the same direction: overparameterized networks can fit random labels perfectly, so generalization cannot be certified from the training procedure alone (Zhang et al., arXiv:1611.03530), and transformer performance on compositional tasks collapses as problem depth exceeds what pattern-matching over the training distribution can span (Dziri et al., arXiv:2305.18654).

That leaves output verification — check every answer against ground truth. Where a checker exists, this works, and Section 5 credits it. But notice what the arrangement concedes: if every output must be independently verified by a sound external procedure, the external procedure is the reasoning system, and the estimator is a proposal generator in front of it. That is a legitimate architecture. It is also an admission that the estimator itself decided nothing. The failure mode of the arrangement is equally well documented — the checker is dropped under schedule pressure, or was never sound, and fabricated citations reach a federal court filing. The sanctions decisions in Mata v. Avianca (S.D.N.Y., 2023) and the case law that has accumulated since are the public record of exactly this collapse.

One more consideration belongs here because it is a property of the ecosystem rather than any single model: the training corpus is not a fixed resource. As model output accumulates in the public text distribution, recursive training degrades the tails of the learned distribution — model collapse, demonstrated formally and empirically (Shumailov et al., Nature, 2024). A verification regime designed against today’s estimator is aimed at a moving target whose drift is driven by the estimators themselves. Deterministic systems have many limitations. Being made worse by their own prior outputs is not among them (BGL Comparative Evaluation CE-19, internal, sealed).

Where the Construction Is Strong

A brief of this kind earns credibility by conceding accurately. The estimator class is genuinely strong wherever its objective and the task coincide — that is, wherever the deliverable is plausible language and an authoritative referent exists to check it against, or no authoritative referent is required at all.

Transduction. Translation, register transformation, formatting, and summarization-for-orientation are tasks where the input fully contains the answer and the work is re-expression. The architecture was invented for translation (Vaswani et al., arXiv:1706.03762) and remains superb at it. The faithfulness caveat of Section 3 still applies — summaries hallucinate (Maynez et al., arXiv:2005.00661) — but for orientation, triage, and first drafts, the residual error is tolerable because a human disposition follows.

Proposal generation against a sound checker. Code drafted by a model is compiled, typed, tested, and reviewed; each stage is an external verifier the model does not control. Mathematical conjecture, query drafting, and search-space pruning share the shape. Where the checker is sound and actually run, the estimator’s error rate prices in as iteration cost rather than institutional risk. The laboratory uses statistical systems in exactly this posture in its own tooling — outside every verdict path, upstream of deterministic gates (BGL Internal Memorandum DI-0004, Annex B, internal, sealed).

Interface. Mapping informal human language onto formal system vocabulary — and formal outputs back into prose — is distributional work, and the estimator is the best artifact ever built for it. Retrieval-augmented arrangements extend this usefully by grounding generation in fetched documents (Lewis et al., arXiv:2005.11401), subject to the positional degradation already noted (Liu et al., arXiv:2307.03172).

Elastic breadth. No deterministic system this laboratory will ever build can converse passably about maritime law, Fortran, and Persian poetry in the same session. Few-shot generality across open domains is real (Brown et al., arXiv:2005.14165) and is not the property under dispute.

The pattern across all four is exact and worth stating once: the estimator is strong where plausibility is the product, or where something else — a compiler, a source document, a human, a derivation engine — bears final responsibility for truth. It is weak precisely where it is currently being marketed hardest: as the component that decides. The strengths are genuine. They are also, without exception, the strengths of an instrument, not of an authority (BGL DI Charter, Annex C, internal, classified).

The Verdict-Path Exclusion

Within this laboratory, a verdict path is any computational path whose terminal output an operator, client, or counterparty is entitled to rely on as a finding of the system — a Stratum governed-reasoning conclusion, a Tektaris operational determination, a derivation record issued under the M.A.D. framework. The exclusion, in force since September 2025 and enforced at review: no component on a verdict path may be a statistical estimator of language, of any size, from any vendor, under any wrapper (BGL Internal Memorandum DI-0004, “Verdict-path admissibility criteria,” internal, sealed).

The rule is not a style preference. It follows from the laboratory’s admissibility criteria, each of which maps to a limit established above. A verdict-path component must be replayable — identical inputs produce identical outputs, attested by hash, run to run and machine to machine (defeated by conditional reproducibility; Section 3). It must be lineage-complete — every output traceable through recorded intermediate structures to source facts (defeated by the absence of any such structures; testimony is not lineage). It must be refusal-capable by construction — structurally unable to emit a conclusion its premises do not support, rather than trained to be unlikely to (defeated by the objective itself; an estimator of plausible text has no premises). And it must be bounded-failure — its failure modes enumerable in advance and detectable at the boundary (defeated by miscalibration and by errors that arrive in the same register as correct output). D.A.E. satisfies these criteria by construction, which is the entire reason it exists; the point of this brief is not that the laboratory’s engine is good but that the estimator class cannot satisfy them even in principle, because each criterion names something the training objective never optimized (BGL Comparative Evaluation CE-19, internal, sealed).

The regulatory environment is converging on the same distinction, more slowly and in gentler language. NIST’s AI Risk Management Framework makes validity, accountability, and transparency the load-bearing trustworthiness characteristics (NIST AI 100-1, 2023), and its generative-AI profile catalogues confabulation as a risk to be managed rather than a defect to be engineered out — an accurate description, and a telling one (NIST AI 600-1, 2024). The EU AI Act imposes traceability, logging, and human-oversight obligations on high-risk systems (Regulation (EU) 2024/1689, Arts. 12–14), obligations a statistical estimator can only meet by being wrapped in exactly the external machinery whose necessity concedes the argument. ISO/IEC 42001 builds the management-system scaffolding around the same gaps (ISO/IEC 42001:2023). And the reliability sector, which cannot outsource consequences, said it plainest: human-in-the-loop, because the systems are not independently verifiable (NERC, 2024). The laboratory reads this corpus as the institutional world discovering, domain by domain, the constructional facts of Sections 2 through 4 — and legislating compensations for a property that deterministic systems simply have.

Two objections deserve answers on the record. The systems will improve. They will — along the axis they are optimized on. Scaling laws govern cross-entropy (Kaplan et al., arXiv:2001.08361; Hoffmann et al., arXiv:2203.15556); none of the criteria above is a function of cross-entropy, and the empirical record shows the reasoning failures tracking distributional statistics at every scale yet reached (McCoy et al., arXiv:2309.13638; Dziri et al., arXiv:2305.18654). Wrap the estimator in verification and use it anyway. On verdict paths, this is the proposal-generator architecture of Section 4, and the laboratory’s objection is not that it fails but that its accounting is dishonest: the verifier is the system; the estimator is upstream tooling; and upstream tooling is precisely where the exclusion already permits it. The exclusion does not ban the instrument. It bans the impersonation of an authority by an instrument (BGL Internal Memorandum DI-0004, internal, sealed).

The Initiative

The Deterministic Intelligence Initiative was chartered in June 2025 (BGL DI Charter, internal, classified). It is a program of BlackGrid Labs. It is classified in its entirety.

The following is the full extent of what the laboratory discloses. The initiative exists. Its subject is deterministic intelligence — reasoning systems that satisfy the admissibility criteria of Section 6 by construction, at scopes not previously attempted. It draws on the M.A.D. framework and the D.A.E. engine already described in the laboratory’s public materials, and on nothing else that will be named. Its findings are recorded in a sealed ledger series (BGL Run Ledger DI-1108 et seq., internal, sealed). Its existence has been disclosed to certain counterparties under instrument; those disclosures are themselves confidential (BGL Counsel Review CR-31, internal, 2026-07).

No further documents in the DI series will be published. Requests for briefings are considered only under agreement, and only where a counterparty’s mandate makes the material relevant: research@blackgridlabs.com.

This brief states what statistical language systems are and what follows from what they are. What the laboratory is doing about it is not a public matter.