Gregory Caldwell Beier
Susarb LLC, Cambridge, Massachusetts
March 2026
(c) 2026 Gregory Caldwell Beier. Licensed under CC BY-NC-ND 4.0.
Financial markets and neural networks appear unrelated. They are studied by different disciplines, evaluated by different standards, and funded through different channels. This paper identifies a structural operation they share: both perform lossy reconstruction from compressed signals, and both fail when the compression is too severe. A price compresses human judgment into a scalar. A training example compresses the reasoning that produced it into a data point. The downstream system - whether a capital allocator or a neural network - must then reconstruct enough of the lost structure to act effectively, and when it cannot, the consequences differ in form but not in kind: capital goes to the wrong place, or the model fails on data it has not seen before. The paper defines Beier Extropy, E_B = H(P,D) - H(P), as the information restored to a compressed signal when structured Disclosure is attached. E_B is formally equivalent to the conditional entropy H(D|P) - a standard quantity. The contribution is not the formula but the identification that this single quantity governs performance on two surfaces simultaneously, through an identical mechanism, because both systems process the same data stream. The companion paper (Beier, 2026c) derives the rate-distortion analysis of price compression formally; the present paper extends that analysis to a second surface and shows that a single intervention - structured disclosure with track record incentives - improves both.
Keywords: information theory, reconstruction, lossy compression, dual surface, Beier Extropy, market microstructure, neural networks, data quality
JEL Codes: D82, D83, G14, C45
Financial markets and neural networks do not usually appear in the same sentence outside of marketing copy. Economics studies how distributed agents coordinate through prices. Machine learning studies how statistical models generalize from training data. The fields share vocabulary - information, signal, noise, efficiency - but the shared vocabulary is understood as loose analogy, not as evidence of shared structure.
This paper argues the analogy reflects something real. Markets and neural networks both receive compressed signals and must reconstruct latent structure from them. They fail for the same reason - too much compression leaves too little structure to recover - and they improve by the same mechanism: enriching the input signal so it carries more of the structure that produced it. The improvement on both surfaces can be measured by a single quantity, and that quantity operates on a shared data stream.
The argument proceeds as follows. Section 2 establishes the common operation of reconstruction from compressed signals, developing the market and AI surfaces in parallel. Section 3 defines Beier Extropy and establishes its properties. Section 4 explains why the parallel is structural rather than metaphorical. Section 5 derives testable predictions. Section 6 addresses objections. Section 7 develops implications for market design, AI architecture, and institutional design. Section 8 states limitations. Section 9 concludes.
Define a reconstruction system as any system S that receives a compressed signal C, derived from a source signal G through a lossy compression function f (so that C = f(G) and H(C) < H(G)), and must produce an action A whose quality depends on information present in G but absent from C. The reconstruction burden of S is the amount of information about G that S must infer from C to produce A at acceptable quality.
A financial market participant is a reconstruction system. G is the full judgment behind the price - the analyst's evidence, methodology, confidence, identity, and scaling assessment. C is the price. A is the investment decision. Decision quality depends on information that was present in G but absent from C.
A neural network is a reconstruction system. G is the full causal process behind the training data - the circumstances, reasoning, and context that produced the observed outcome. C is the training example. A is the prediction on new data. Generalization quality depends on causal structure that was present in G but absent from C.
The abstraction captures both systems without distortion. Conant and Ashby (1970) proved that every good regulator of a system must be a model of that system - reconstruction is necessary, not merely useful, for effective action on compressed signals. The claim is not that markets and neural networks are the same system. They differ in every implementational detail: markets involve strategic agents, neural networks run on gradient descent; markets settle into equilibria, networks navigate loss landscapes. The claim is that they share one specific operation - reconstruction from compressed signals - and that this operation responds to the same intervention: enriching the input signal.
The relationship is analogous to that between a telescope and a microscope. Different instruments applied to different domains. They share a single optical operation - focusing electromagnetic radiation through lenses - and improvements in lens quality improve both instruments for the same reason, through the same mechanism, measurable by the same quantity. No one would claim a telescope is a microscope. But no one would deny they share an optical principle.
The companion paper on lossy compression (Beier, 2026c) derives the rate-distortion analysis of price formation within a CARA-normal framework. Four results characterize the compression. The price system's distortion equals the residual variance on the consensus value dimension plus the total prior variance on all constraint dimensions (Proposition A). The minimum achievable distortion at the same compression rate, computed via reverse water-filling, is strictly lower (Proposition B). The excess distortion is strictly positive, increasing in the number of constraint dimensions, and persistent even in the limit of perfect market efficiency (Proposition C). Supplementary disclosure bits produce strictly higher marginal distortion reduction than additional price-precision bits (Proposition D).
This section develops the economic consequences rather than re-deriving the formal results.
The price of a barrel of oil compresses an extraordinarily rich information environment - the reasoning of every analyst who contributed to the figure, their methodologies, their confidence levels, their assessments of what happens when conditions change - into a number. The history of financial innovation has deepened this compression at every stage. Coinage stripped identity from exchange. Bills of exchange stripped proximity. Derivatives stripped the underlying asset. Securitization stripped the individual loan. Algorithmic trading stripped human deliberation. At each stage the signal became faster, more liquid, and carried less of the judgment that produced it. This progression was not a mistake - scale demanded anonymity, anonymity demanded compression - but it imposed a specific cost: every downstream user of the price must reconstruct enough of the lost information to make their own decision.
This reconstruction is what a substantial fraction of the financial services industry does. Sell-side research analysts reconstruct corporate strategy from financial statements and price movements. Credit rating agencies reconstruct default probability from compressed financial metrics. ESG rating providers reconstruct sustainability performance from standardized fact sheets. Burton-Taylor Consulting (2025) reports $44.3 billion in direct market data spending; the full reconstruction burden - including analyst salaries, compliance infrastructure, and computational resources - likely exceeds several hundred billion dollars annually.
The 2008 financial crisis demonstrated what happens when reconstruction fails at scale. Mortgage-backed securities were structured to be what Dang, Gorton, and Holmstrom (2012) call "information-insensitive" - deliberately compressed so that buyers need not evaluate the underlying assets. When the compressed signal could no longer be trusted, the market was forced to confront what had been stripped away, and the reconstruction infrastructure collapsed under the load. Bernanke (1983) documented the same pattern in the 1930s: the failure of financial intermediaries destroyed the institutional capacity for reconstructing borrower quality, prolonging the Depression.
The same framework applies to neural network training data, though machine learning has not described it in quite these terms.
A training example is a compression of the process that produced it. When a dataset records that a particular loan defaulted, that single fact compresses a rich causal environment - the borrower's circumstances, the lender's reasoning, the economic conditions, the regulatory context, the chain of decisions that led to the loan being issued. What the training example keeps is the outcome. What it loses is everything that would explain why.
The neural network's task is to reconstruct enough of the causal structure from the compressed examples to generalize to new situations. When training examples carry rich contextual information - a low compression ratio - the network can recover structure effectively. When they are thin records stripped of reasoning, the network has nothing to work with except surface correlations, and it learns to mimic patterns rather than to reason.
Tishby, Pereira, and Bialek (1999) formalized learning as optimal compression through the Information Bottleneck principle, studying the compression the network performs internally during learning. The present paper studies a different compression point: the compression of the signal the network receives as input, before any internal processing occurs. Both are information-theoretic. They operate at different points in the pipeline.
Vapnik and Vashist (2009) provided the most direct formalization of the AI surface claim. Their Learning Using Privileged Information framework addresses the case where training data is enriched with additional information available during training but not at test time - the "teacher's explanation" that carries what the present paper calls the reasoning behind the data point. Vapnik proves that models trained with privileged information converge faster and generalize better. The present paper adds the identification that the "privileged information" is simultaneously useful to human market participants - the dual-surface claim.
The persistent failure mode of financial AI - models that perform well in backtesting and fail in deployment - follows from the compression framework. The backtesting environment shares the statistical surface of the training data. Deployment demands generalization to conditions the model has not seen, where the causal reasoning behind historical prices matters more than the prices themselves. If that reasoning was compressed out of the training signal, the model has no basis for generalization. Zhang et al. (2017) confirmed that neural networks can memorize random labels without generalizing - learning surface patterns is possible without learning structure. Peters, Janzing, and Scholkopf (2017) proved formally that causal models generalize under distribution shift while purely statistical models do not, giving the AI surface prediction theoretical teeth beyond information-theoretic intuition.
Ng (2021) articulated the practical research program: data-centric AI, where the binding constraint on model performance is data quality rather than model architecture. The present paper provides the information-theoretic explanation for why data quality matters (the compression ratio of the input signal determines the reconstruction quality the model can achieve) and connects the AI data quality problem to the market transparency problem through a shared mechanism.
Define Beier Extropy as:
E_B = H(P,D) - H(P)
where P is the original compressed signal (the price, the training example) and D is structured Disclosure data attached to it. E_B is the information restored to the signal when Disclosure is present, measured in bits.
E_B is formally equivalent to the conditional entropy H(D|P) by Shannon's chain rule. This is a standard information-theoretic quantity. The contribution is not the formula but the identification that this single quantity governs performance on two surfaces - market coordination and AI training - through an identical mechanism, because both surfaces process the same data stream.
A terminological note. The word "extropy" was introduced by Tom Bell (writing as T.O. Morrow) in 1988 and developed by Max More (2003) as a philosophical principle denoting the inverse of entropy in a vitalist sense - the tendency of living and intelligent systems toward greater order, complexity, and information. Beier Extropy is a distinct concept: a specific, formally defined information-theoretic quantity measuring the bits restored to a compressed signal by structured metadata attachment. The terminological resonance is acknowledged, but the concepts share a word rather than a definition.
Three properties require clarification.
E_B is trivially positive for any metadata attachment - adding any data to a signal increases its entropy. A random string appended to a price also increases H(P,D). What prevents E_B from being vacuous is the architecture behind D. The structured Disclosure that produces D is designed so that the information it carries is informative about the underlying judgment J. Five structured fields - Input (evidence consulted), Method (analytical framework), Output (conclusion with confidence), Identity (analyst with track record), and Scaling (assessment of interaction effects at aggregate scale) - ensure that D carries judgment-relevant information rather than noise. Identity creates reputational stakes. Output creates timestamped, falsifiable claims that enable track record accumulation. The verification architecture converts potential information gain into actual coordination improvement.
E_B measures what Disclosure adds, not how much of the original judgment is recovered. The full judgment J has entropy H(J). Disclosure D captures some of that judgment but not all - no finite set of structured fields carries the full richness of a human decision process. E_B is therefore strictly less than the original information loss L = H(J) - H(P). It is a partial reversal. What makes the partial reversal significant is that the marginal value of the first bits restored is very high. A price that carries methodology and confidence is enormously more useful than a bare scalar, even though it still does not carry the analyst's full decision state. The companion paper (Beier, 2026c, Proposition D) proves this formally: at the current operating point, disclosure bits produce strictly higher marginal distortion reduction than additional price-precision bits.
E_B operates on both surfaces through the shared data stream. When a market participant attaches structured Disclosure to a transaction, that Disclosure is simultaneously available to the buyer (the market surface) and to any AI system training on the resulting data (the AI surface). The enrichment is applied once and operates on two surfaces because both process the same data. Jones and Tonetti (2020) formalized the enabling property: data is nonrival. The same Disclosure serves the buyer and the AI simultaneously without being consumed by either.
Many connections between economics and machine learning are metaphorical. Economists speak of markets "learning." ML researchers speak of "market" mechanisms in multi-agent systems. These are useful analogies but not structural identities.
The dual-surface claim is of a different kind. It asserts that a single information-theoretic operation - reconstruction from a compressed signal - is literally performed by both systems, and that the same intervention (enriching the signal's information content) improves both for the same reason. E_B does not measure two different things that happen to share units. It measures one thing - how much less compressed a signal becomes when Disclosure is attached - and that one thing governs performance in both domains because both domains draw on the same physical data stream.
The Disclosure data D enters both systems. A structured five-field assessment attached to a financial transaction is readable by the human buyer and parseable by the AI training pipeline. The enrichment is not applied twice. It is applied once and propagates to both surfaces through the shared data.
The implication is that improving market transparency and improving AI training data are not two separate projects. They are one project. A structured Disclosure that enriches a price for a human buyer enriches the training corpus for AI systems at the same time. The two surfaces are coupled through the data stream they share.
The dual-surface identification can be stated as a formal result.
Proposition (Simultaneous improvement). Let D be a structured disclosure attached to compressed signal P about a K-dimensional constraint state V. Define:
Market surface improvement: Delta_market = Var(V|P) - Var(V|P,D)
AI surface improvement: Delta_AI = E[l*(P)] - E[l*(P,D)]
where l*(S) is the Bayes-optimal prediction loss for a model observing signal S. Then:
(i) Delta_market > 0 if and only if I(V; D | P) > 0 - the disclosure carries information about the constraint state beyond what the price contains.
(ii) Delta_AI > 0 if and only if I(V; D | P) > 0 - the same condition.
(iii) Therefore Delta_market > 0 if and only if Delta_AI > 0: the two surfaces improve simultaneously, and both improve if and only if the disclosure carries constraint information the price does not.
Proof. (i) Var(V|P) - Var(V|P,D) > 0 iff the posterior on V given (P,D) is tighter than given P alone. Under the Gaussian framework, this holds iff D carries mutual information about V beyond what P carries: I(V; D|P) > 0. This is equivalent to H(V|P) > H(V|P,D), which holds iff D is not conditionally independent of V given P.
(ii) The Bayes-optimal loss l*(S) is a function of the posterior distribution of V given S. A tighter posterior (lower entropy) produces lower Bayes-optimal loss for any proper loss function. Since I(V; D|P) > 0 implies H(V|P,D) < H(V|P), the Bayes-optimal loss under (P,D) is strictly lower than under P alone.
(iii) Both conditions reduce to I(V; D|P) > 0. The disclosure enriches both surfaces simultaneously because both compute posteriors over the same V using the same data (P,D). The enrichment is applied once and operates on two surfaces because both surfaces process the same data stream. QED.
The proposition formalizes the dual-surface identification: E_B = H(D|P) is the total information the disclosure adds to the signal, and I(V; D|P) <= H(D|P) is the portion relevant to the constraint state. Whenever I(V; D|P) > 0, both surfaces improve. The condition fails only when D is pure noise or is redundant with what P already carries.
The dual-surface claim generates predictions falsifiable on each surface independently and jointly.
Market surface. Sectors adopting structured Disclosure should show measurable declines in information reconstruction costs - analyst coverage per unit of capital deployed, compliance expenditure as a share of transaction volume - within five years of adoption. Price discovery should be faster and more accurate, measurable through bid-ask spread reduction and price error relative to subsequent realized outcomes. The magnitude of improvement should be greatest in markets with the highest current compression ratio: structured products, infrastructure investments, and emerging market sovereign debt, where the gap between the complexity of the underlying asset and the dimensionality of the price signal is largest.
AI surface. Models trained on Disclosure-enriched datasets should demonstrate superior out-of-sample performance compared to models trained on equivalent volumes of price-only data. The improvement should be most pronounced in tail-event prediction, where surface patterns from normal conditions are least informative and the reasoning structure behind the data matters most. Models trained on Disclosure data should exhibit better calibration - their stated confidence should more closely track their actual accuracy - because the training data itself carries calibrated confidence assessments scored by proper scoring rules (Gneiting and Raftery, 2007).
Cross-surface. Markets served by AI systems trained on Disclosure-enriched data should exhibit better capital allocation than markets served by AI trained on price-only data, even controlling for the direct effect of Disclosure on human participants. An autocatalytic dynamic should emerge: better Disclosures train better AI, better AI routes capital to better judgment, better-funded judgment produces better Disclosures. This dynamic should be observable as increasing E_B over time within adopting sectors, with the rate of increase accelerating as the feedback loop establishes itself.
The formula is trivial. E_B = H(P,D) - H(P) is a standard entropy difference. Any information theorist could write it down. Correct. The formula is not the contribution. The contribution is the identification that this formula, applied to structured judgment metadata, operates simultaneously on market coordination and AI training through an identical mechanism. Writing down H(D|P) is trivial. Recognizing that it measures the same intervention on two surfaces is not.
Markets and neural networks are too different for a meaningful structural parallel. Valid against any claim that markets and neural networks are the same system. Not valid against the narrower claim that both share one specific operation (reconstruction from compressed signals) and that this operation responds identically to the same intervention (signal enrichment). Sargent (1993) modeled economic agents explicitly as statistical learners that form beliefs from data and update them over time, establishing the formal bridge between bounded rationality and machine learning that the dual-surface claim extends.
More information can sometimes worsen coordination. Morris and Shin (2002) showed that public information's value depends on how agents use it - when agents overweight public signals relative to private signals, additional public information can reduce welfare. The response: Morris-Shin applies to undifferentiated public signals. Structured Disclosure carries identity and track record, which allows the market to weight signals by demonstrated quality rather than overweighting all public signals equally. Angeletos and Pavan (2007) confirmed that the welfare implications of public information depend on quality differentiation, and quality-differentiated information restores the social value of public disclosure.
The AI surface claim is empirical and might be false. Correct - and the prediction is stated in falsifiable form. If models trained on Disclosure-enriched data do not outperform models trained on price-only data, the AI surface prediction fails. But the information-theoretic argument is strong, and Vapnik's privileged information framework (2009) already demonstrates the mechanism in controlled settings.
The cross-surface coupling could produce a negative feedback loop. Acemoglu, Kong, and Ozdaglar (2026) formalize conditions under which AI substituting for human judgment degrades collective knowledge - what they call knowledge collapse. If AI trained on Disclosure data becomes a substitute for rather than a complement to human judgment production, the Disclosure stream could thin over time. The prediction of an autocatalytic positive feedback assumes the disclosure mechanism maintains the incentive for human judgment production (through track record capital) even as AI capability grows. Whether this assumption holds is an empirical question the cross-surface predictions are designed to test.
Market transparency is not merely a regulatory virtue or a governance preference. It is an information-theoretic intervention that reduces the reconstruction burden on every participant. The companion paper (Beier, 2026c) proves that the price system operates below the rate-distortion bound - that it introduces more distortion than the minimum achievable at its compression rate. Structured Disclosure moves the system toward the bound by preserving information at source that the price would otherwise compress away. The economic cost of the intervention (producing Disclosure) can be compared directly to the cost it eliminates (reconstruction expenditure), and the companion paper on excess distortion (Beier, 2026h) argues that the latter vastly exceeds the former because Disclosure is produced once while reconstruction is performed by every downstream user.
A practical precedent exists. XBRL (eXtensible Business Reporting Language) already provides structured financial data attached to regulatory filings. But XBRL structures facts - revenue, assets, liabilities - not judgment. The five-field Disclosure format structures reasoning, methodology, confidence, and scaling assessment. XBRL enriches the data layer; Disclosure enriches the judgment layer. The distinction corresponds to the information-theoretic distinction between preserving the low-dimensional consensus information (which XBRL does and the price already approximates) and preserving the high-dimensional constraint information (which Disclosure does and the price structurally cannot). Berg, Koelbel, and Rigobon (2022) documented the consequence of structuring facts without judgment: ESG ratings diverge dramatically across providers, with correlations as low as 0.38, because each provider reconstructs different judgment from the same structured facts. Disclosure addresses the divergence at source by providing the scored judgment that the fact-based ratings attempt to reconstruct.
The quality of economic AI depends not only on model architecture but on the information content of the training signal. Training financial AI on historical price data is, in information-theoretic terms, training on maximally compressed signals and expecting the model to reconstruct the causal structure that was compressed out. The problem is not the model - it is the data the model was given. Better architecture cannot compensate for a training signal that has had its causal structure stripped out. Gebru et al. (2021) proposed Datasheets for Datasets and Mitchell et al. (2019) proposed Model Cards, both of which attach structured metadata to training data - documenting provenance, composition, and intended use. These are partial implementations of Disclosure on the AI surface. The present framework generalizes both by connecting data documentation to market transparency through a single measure.
The implication is that the AI community's data-centric turn (Ng, 2021; Zha et al., 2023) and the financial regulatory community's push for structured disclosure are converging on the same intervention from opposite directions. Neither community has recognized the convergence, because the institutional separation between economics and machine learning prevents the shared mechanism from being visible. E_B makes it visible.
The distinction between "improving markets" and "improving AI" is artificial when both systems draw on the same data stream. An institution that invests in market transparency is, whether it knows it or not, also investing in AI capability. The reverse is equally true. The two investments are the same investment, visible on two surfaces.
This has a practical consequence for resource allocation. Regulatory agencies that mandate structured disclosure (CSRD, TCFD, SEC climate rules) are currently evaluated on their market transparency benefits alone. The AI training benefit - the improvement in model quality when AI systems train on the richer data stream the regulation produces - is an uncounted positive externality of the regulation. Conversely, AI companies that invest in curating high-quality, well-documented training data are producing a public good for market participants who could use the same documentation to reduce their reconstruction burden. Neither side of this externality is currently priced. E_B provides the measure that would allow it to be.
The dual-surface claim rests on the assumption that the same Disclosure data D is useful to both human market participants and AI training pipelines. This assumption is plausible - structured text with attributed confidence levels is readable by both - but the degree to which each surface benefits from the same data may differ. Human participants may weight identity and narrative reasoning; AI systems may weight the quantitative confidence assessments and outcome scores. The enrichment measured by E_B is the total information added to the signal, but the useful fraction may differ across surfaces. A surface-specific analysis decomposing E_B into the components most valuable on each surface would strengthen the framework but is beyond this paper's scope.
The autocatalytic cross-surface prediction (better Disclosure trains better AI, which routes capital to better judgment, which produces better Disclosure) assumes the feedback loop is positive. If AI substitutes for human judgment production rather than complementing it - if better AI reduces the incentive to produce human Disclosures - the loop could be negative, producing the knowledge collapse that Acemoglu, Kong, and Ozdaglar (2026) formalize. The Disclosure Incentive's track record mechanism is designed to maintain the human judgment production incentive, but whether it does so as AI capability grows is an empirical question.
The market surface predictions rely on the companion papers' formal apparatus, particularly the rate-distortion analysis (Beier, 2026c) and the incentive compatibility results (Beier, 2026r). If the formal foundations are revised, the market surface predictions must be updated accordingly.
The paper does not address the cost of producing structured Disclosure. The companion paper on the Disclosure Incentive (Beier, 2026b) develops the economics of Disclosure production as an investment in track record capital, but the production cost enters the welfare analysis: the net benefit of Disclosure is E_B minus the cost of producing D, and this net benefit varies by domain, agent, and institutional context.
Markets and neural networks both reconstruct structure from compressed signals. They fail for the same reason - the signal carries too little of the structure that produced it - and they improve by the same mechanism: enriching the input so it carries more of its own context.
Beier Extropy, E_B = H(P,D) - H(P), is formally simple. What makes it worth naming is that it measures the enrichment of a signal processed by two different systems - one institutional, one computational - through what turns out to be the same operation. The enrichment is applied once. It operates on two surfaces because both draw on the same data.
The predictions are testable. Markets that adopt structured Disclosure should show reduced reconstruction expenditure and improved price discovery. AI models trained on Disclosure-enriched data should generalize better, particularly in tail events, and should exhibit better-calibrated confidence. If the cross-surface coupling is real, market transparency and AI capability should reinforce each other over time through the shared data stream.
If the predictions hold, the implication is that economic coordination and machine learning are not merely related fields that can borrow from each other but two expressions of the same underlying problem: how to reconstruct structure from compressed signals. Improving the signal improves both, and the separation between the two fields turns out to be institutional rather than fundamental.
This paper presents results from the Prolegomena to the Unification of Economics, Computer Science, Mathematics, and Physics (Beier, forthcoming 2026). The dual-surface identification developed here connects the rate-distortion analysis of price compression (Beier, 2026c) to the economics of AI training data quality through a shared information-theoretic operation. The author invites engagement from the information economics, machine learning, and market microstructure communities.