How Good Can a Zero-Edge Backtest Look After Strategy Search?

Suppose each candidate strategy produces independent Normal daily returns with a true mean of zero. Give every strategy 252 daily observations, calculate its annualized sample Sharpe ratio, test 100 independent strategies, and report only the best one.

Under that null, the median winning Sharpe is 2.48. There is a 90.52% chance that the winner reaches an annualized Sharpe of at least 2 even though every candidate has zero true expected return.

Under the stated Normal zero-mean null, the usual t statistic for a sample mean follows a Student t distribution with T - 1 degrees of freedom. If the backtest has T observations and the Sharpe ratio is annualized with P periods per year, the annualized sample Sharpe is the t statistic multiplied by sqrt(P / T).

If F is the resulting single-strategy Sharpe cumulative distribution and N strategies are independent, then the best result has cumulative distribution F(x)^N. That order-statistic identity lets Grizzly Bulls calculate the full winner distribution directly without Monte Carlo noise.

The comparison between one year and three years is important. With 100 independent trials, moving from 252 to 756 daily observations cuts the median null winner from Sharpe 2.48 to 1.42. The chance of a null winner reaching Sharpe 2 falls from 90.52% to 2.77%.

Longer histories reduce sampling noise, but they do not erase the selection problem. If the research process searches many variants and reports the winner, the winner should not be evaluated as though it had been specified before the search began.

The separate out-of-sample null benchmark follows that selected winner into a genuinely independent holdout and shows why an inflated in-sample Sharpe can regress sharply even when the underlying zero-edge process has not changed.

Real strategy variants are usually correlated, so 100 parameter combinations rarely behave like 100 fully independent hypotheses. Correlation reduces the effective number of independent trials. Real returns can also have serial dependence, skewness, volatility clustering, fat tails, changing regimes, and execution frictions that this benchmark intentionally excludes.

The study therefore does not estimate the false-positive probability of a particular real research process. It provides a transparent reference case that makes one point visible: the strongest result from a search has a different null distribution from a strategy chosen in advance.

Use the multiple-testing calculator for family-wise false-positive arithmetic, the Probabilistic Sharpe Ratio tool for finite-sample Sharpe uncertainty, and the backtest validation stack for the broader research workflow.

The published grid covers 126, 252, 756, and 1,260 return observations crossed with 1, 5, 20, 100, and 500 independent trials. For each cell it reports the median, 95th percentile, and 99th percentile of the best annualized Sharpe plus the chance that the winner reaches Sharpe 1, 1.5, or 2.

Download CSV · Download JSON

This study is available for factual citation and reuse. Preserve the stated assumptions and limitation when they materially affect interpretation, and use the stable canonical URL rather than a temporary search or distribution link.