OPEN-SOURCE QUANTITATIVE FALSIFICATION ENGINE

PENROSE

Most edges don't survive.
Penrose validates the few that do.

An open-source, falsification-first research engine for quantitative trading claims. At its heart, a power-aware referee: it reconstructs the strategy from scratch, then vets the reconstruction and its track record against a series of quality gates — is the edge real, or just an artifact of how it was found? — and reports what survives: kill, underpowered, watch, or research-supported. And it doesn't just judge — it learns: distilling reusable principles from what fails, then generating new candidate hypotheses from those lessons, each earning its own verdict through the same gauntlet. It finds no alpha of its own — only an honest, compounding account of what survives.

9
Pipeline stages — P1 through the human-only P9 gate
4
Verdicts: kill · underpowered · watch · research-supported
13
Safety invariants — explicit public system boundaries
1
Holdout — single-use, permanently locked
WHY "PENROSE"
Almost everything that falls in is lost.
The Penrose process is how energy escapes.

A black hole traps nearly all the matter and light that crosses its horizon — gone, unrecoverable. Yet Roger Penrose showed there is a way for energy to escape a spinning black hole, carrying real information back out with it.

Quant research has the same shape. Nearly every backtested edge gets pulled into the void — selection bias, overfitting, look-ahead, decay — and dies there. Penrose is the test almost nothing survives. The rare claim that escapes the gauntlet has earned the energy it carries out. We named the engine after the escape — even though the void is most of the work.

THE ETHOS
We try to kill every edge.
The few that survive earn their trust.

Most backtests are stories the data tells about itself — selection bias, overfitting, and look-ahead dressed up as alpha. Penrose assumes an edge is noise until it survives deflation, out-of-sample testing, and a single-use locked holdout. When it can't tell, it says so — underpowered, not dead. The point isn't to be right; it's to not fool yourself — in the open, where anyone can check the work.

THE METHOD · THREE NUMBERS THAT KEEP US HONEST

A harness built to catch its own mistakes

DSR
Deflated Sharpe Ratio
Sharpe, penalized for how many strategies were tried. A high deflated value means the result is unlikely to be a lucky draw from the search — the threshold is sample-dependent, not a magic line.
residual_sharpe
beta-adjusted
Risk-adjusted return after market beta is stripped out. The honest measure of selection skill — what is left when "just long the market" is removed.
beta
exposure
How much of a return is simply riding the market. High beta with low residual is exposure wearing the costume of skill. We always report both, side by side.
The harness caught a real one.
A v0 thesis pipeline looked elite in-sample — gross Sharpe 2.10 — and quietly hurt out-of-sample. The beta-adjusted OOS harness flagged the overfit before a dollar of paper followed it. Catching that is the whole point.
WHAT MAKES IT DIFFERENT

Not a backtester. A referee.

Backtesters ask "can I find an edge?" Penrose asks the harder question: "after counting the whole search and trying hard to break it, is there credible evidence left?" It referees a claim — from a paper, a generator like RD-Agent or Qlib, a notebook, or yourself — not just your own strategy.

Tools like Qlib generate factors and backtest them well — but none of them count the size of the search, deflate for it, or lock a holdout. Penrose is the layer that does.

01 · INGEST
Claim
A paper, a generated factor, or a thesis — sanitized, treated as untrusted.
02 · REBUILD
Faithful reconstruction
Rebuilt in a sandbox; reconstruction fidelity is itself an evidentiary risk.
03 · ATTACK
Adversarial testing
Deflation, walk-forward, regime splits, bootstrap, permutation, locked holdout.
04 · JUDGE
Calibrated verdict
kill · underpowered · watch · research-supported — each with a minimum detectable effect.
Referees third-party claims — papers, RD-Agent / Qlib output, not just your own code.
Counts the whole search — discarded candidates stay in the deflation denominator.
Single-use locked holdout — read once, never peeked at during testing.
Separates kill from underpowered — "can't tell" is never "it's dead."
Measures its own detector — placebo, injection, and a 5-null calibration battery.
Quarantines generated code in a sandbox; a human approves before anything is trusted.
PROOF · THE RARE SURVIVOR

What it takes to survive the referee

"Stocks tend to repeat their own dividend-month returns — a calendar seasonality in the cross-section." — one of 212 published anomalies we put through the referee.
deflated vs all 212 searched ✓ 3-fold sign-stable ✓ survives regime split ✓ single-use holdout ✓ most of the other 211: killed / underpowered
RESEARCH-SUPPORTED Cleared deflation against all 212, three-fold sign-stability, a regime split, and a single-use holdout — only 6 of 212 got this far. That's the whole point: almost nothing survives. "Research-supported" means it passed the referee's bar as a candidate to watch — not a profit forecast, and nothing promotes without a human.

Run it yourself.

Penrose is a research release — an independent, power-aware falsification referee for quantitative trading claims. Clone the repo, install, and point it at your experiments.

$ git clone https://github.com/PattersonResearch/Penrose
$ pip install -e .

Open source, Apache-2.0 — contributions welcome: add a data adapter, a strategy module, or a new calibration null. Issues and PRs on GitHub.

See what survives.

An independent, power-aware referee for quant claims. Test a strategy before you trust it — and find out how little holds up.