OPEN-SOURCE QUANTITATIVE FALSIFICATION ENGINE

PENROSE

Most edges don't survive. Penrose finds the few that do.

An independent, power-aware referee for quantitative trading claims. It reconstructs a strategy, deflates it for the size of the whole search, tests it out-of-sample against a locked holdout, and reports what survives — kill, underpowered, or supported. It finds no alpha of its own; it measures how little holds up, honestly.

137
Experiments — every one counted in the deflation
4
Verdicts: kill · underpowered · watch · supported
6
Research modules — paper-only, no live trading
1
Holdout — single-use, permanently locked
WHY "PENROSE"
Almost everything that falls in is lost.
The Penrose process is how energy escapes.

A black hole traps nearly all the matter and light that crosses its horizon — gone, unrecoverable. Yet Roger Penrose showed there is a way for energy to escape a spinning black hole, carrying real information back out with it.

Quant research has the same shape. Nearly every backtested edge gets pulled into the void — selection bias, overfitting, look-ahead, decay — and dies there. Penrose is the test almost nothing survives. The rare claim that escapes the gauntlet has earned the energy it carries out. We named the engine after the escape — even though the void is most of the work.

THE ETHOS
We try to kill every edge.
The few that survive earn their trust.

Most backtests are stories the data tells about itself — selection bias, overfitting, and look-ahead dressed up as alpha. Penrose assumes an edge is noise until it survives deflation, out-of-sample testing, and a single-use locked holdout. When it can't tell, it says so — underpowered, not dead. The point isn't to be right; it's to not fool yourself — in the open, where anyone can check the work.

THE METHOD · THREE NUMBERS THAT KEEP US HONEST

A harness built to catch its own mistakes

DSR
Deflated Sharpe Ratio
Sharpe, penalized for how many strategies were tried. A high deflated value means the result is unlikely to be a lucky draw from the search — the threshold is sample-dependent, not a magic line.
residual_sharpe
beta-adjusted
Risk-adjusted return after the crypto market beta is stripped out. The honest measure of selection skill — what is left when "just long the market" is removed.
beta
exposure
How much of a return is simply riding the market. High beta with low residual is exposure wearing the costume of skill. We always report both, side by side.
The harness caught a real one.
A v0 thesis pipeline looked elite in-sample — gross Sharpe 2.10 — and quietly hurt out-of-sample. The beta-adjusted OOS harness flagged the overfit before a dollar of paper followed it. Catching that is the whole point.
WHAT MAKES IT DIFFERENT

Not a backtester. A referee.

Backtesters ask "can I find an edge?" Penrose asks the harder question: "after counting the whole search and trying hard to break it, is there credible evidence left?" It referees a claim — from a paper, a generator like RD-Agent or Qlib, a notebook, or yourself — not just your own strategy.

Tools like Qlib generate factors and backtest them well — but none of them count the size of the search, deflate for it, or lock a holdout. Penrose is the layer that does.

01 · INGEST
Claim
A paper, a generated factor, or a thesis — sanitized, treated as untrusted.
02 · REBUILD
Faithful reconstruction
Rebuilt in a sandbox; reconstruction fidelity is itself an evidentiary risk.
03 · ATTACK
Adversarial testing
Deflation, walk-forward, regime splits, bootstrap, permutation, locked holdout.
04 · JUDGE
Calibrated verdict
kill · underpowered · watch · supported — each with a minimum detectable effect.
Referees third-party claims — papers, RD-Agent / Qlib output, not just your own code.
Counts the whole search — discarded candidates stay in the deflation denominator.
Single-use locked holdout — read once, never peeked at during testing.
Separates kill from underpowered — "can't tell" is never "it's dead."
Measures its own detector — placebo, injection, and a 5-null calibration battery.
Quarantines generated code in a sandbox; a human approves before anything is trusted.
PROOF · ONE THAT GOT OUT

An edge that beat the check

"Stocks tend to repeat their own dividend-month returns — a calendar seasonality in the cross-section." — one of 212 published anomalies we put through the referee.
deflated vs all 212 searched ✓ 3-fold sign-stable ✓ survives regime split ✓ single-use holdout ✓ most of the other 211: killed / underpowered
RESEARCH-SUPPORTED Out-of-sample Sharpe ≈ 1.7 after the full multiple-testing penalty — only 6 of 212 survived. That's the whole point: almost nothing escapes, and Penrose tells you which.

Run it yourself.

Penrose is a research release — an independent, power-aware falsification referee for quantitative trading claims. Clone the repo, install, and point it at your experiments.

$ git clone https://github.com/PattersonResearch/Penrose
$ pip install .

Open source, Apache-2.0 — contributions welcome: add a data adapter, a strategy module, or a new calibration null. Issues and PRs on GitHub.

See what survives.

An independent, power-aware referee for quant claims. Test a strategy before you trust it — and find out how little holds up.