Reports / ML strategy attribution

ML Strategy — Attribution & Hindsight-Bias Audit

Where the returns actually come from, why the headline number is flattering, and whether the algorithm is quietly cheating. Generated 4 November 2026 from the strategy's own walk-forward and live-PnL artifacts.

+52.6%

reported, 27 days

not a capital-base return

+9.3%

same trades, 5 cohorts

one unit of capital

+15.1%

selection alpha / yr

walk-forward, gross, t=2.47

2.6

effective bets

behind a 76.9% win rate

Short version. The algorithm is not look-ahead cheating — the feature pipeline and the walk-forward harness are clean. The distortion is in the reporting: the headline compounds 13 overlapping 5-day windows as if they were sequential, and it is gross of costs. On a real capital base the same trades are worth roughly +9%, not +52.6%. The underlying long-only edge is real but modest: about half of it is European equity beta, and the selection alpha (+15.1%/yr, t=2.47) shrinks to ~+4.6%/yr at 20 bps/leg.

1. What the strategy actually does

A gradient-boosted model ranks 50 EU/Scandi large caps on 52 technical and macro features and buys the top 10 for a 5-day holding period. Two arms are tracked: a no-volume headline arm and a volume-pruned A/B arm. The short leg was removed — the walk-forward showed it losing money consistently — so the live book is long-only. Sizing is discrete half-Kelly with a hard 3% sleeve cap; VAR is halted at zero allocation because its Kelly edge is negative.

Sources: ml_daily_signals.py, ml_daily_signals_novol.py, ml_model.py, data/kelly_sizing.json.

2. Attribution — where the return comes from

Walk-forward, 91 non-overlapping baskets, 2 Sep 2024 → 8 Jun 2026, 50 names, top-10 long-only, gross of costs.

	Cumulative	Annualised	Sharpe	Max DD
Long-only book	+66.9%	+32.8%	2.04	-13.6%
Equal-weight universe	+29.8%	+15.5%	—	—
Selection (excess)	+28.9%	+15.1%	—	—

Half the return is the market. Beta to the equal-weight universe is 1.06 and correlation is 0.84. The universe itself returned +15.5%/yr over the sample; the model's stock selection added +15.1%/yr on top (t = 2.47, positive in 61.5% of baskets). That is a genuine signal — but it is a high-beta long book with a modest selection overlay, not a market-neutral alpha engine.

Cost sensitivity

Costs are the difference between a good strategy and a marginal one. Each basket turns over completely, so the drag compounds quickly.

Cost per leg	Cumulative	Annualised	Sharpe	Max DD
0 bps	+66.9%	+32.8%	2.04	-13.6%
10 bps	+52.4%	+26.3%	1.69	-14.2%
20 bps	+39.2%	+20.1%	1.34	-15.0%

Stability by year

Positive in all three calendar slices, but the sample is short and 2024/2026 are partial years.

Period	Cumulative	Sharpe	Win rate	Baskets
2024 (Sep-Dec)	+7.0%	1.61	52.9%	17
2025	+23.2%	1.46	61.5%	52
2026 (Jan-Jun)	+15.7%	2.19	63.6%	22

What the model is actually keying on

Permutation importance (R² drop) on the last 20% of dates. In-sample and therefore indicative only — but the pattern is unambiguous: macro regime and long-horizon trend, not short-term technicals.

Five of the top ten features are market-wide macro variables (yield curve, VIX, HY spread) and three are long-horizon trend measures (12-month return, distance from 52-week low, 100-day average). The short-term oscillators that dominate the feature list — RSI, MACD, 5-day distances — rank near the bottom. This is a regime-conditioned momentum model, which is exactly the kind of model that can memorise a single macro regime.

Concentration

27 of 50 names contributed positively; the top five names produced 41.5% of gross positive contribution (ERIC-B.ST, PKN.WA, DNP.WA, GLEN.L, AIR.PA). The book is not diversified across the universe — it is a concentrated bet on a handful of names that happened to work.

3. Why the headline looks so good

The live no-volume series reports +52.6% over 27 calendar days. That number is a product of 13 overlapping 5-day windows, originated on consecutive days (median gap: 1 day). Each day's PnL is therefore counted about five times.

Basis	Return	What it means
Reported (compounded overlapping windows)	+52.6%	each 5-day window multiplied as if it followed the last
1 cohort, capped	+11.5%	one unit of capital, at most one basket at a time
5 cohorts, capped	+9.3%	the configured design: five cohorts, 1/5 of the sleeve each

The arithmetic is the story. +52.6% over 27 days implies an annualised rate of roughly +30,271%. No strategy does that; the number is an artefact of compounding overlapping windows. The same trades, on a real capital base, are worth +9% to +12%.

Three further amplifiers:

It is a hot streak. The best three windows contribute +34.4% of the +52.6% total. Remove them and the run is ordinary.
The sample is tiny. The Kelly file itself computes n_eff = 2.6: 13 observations are worth about 2.6 independent bets. A 76.9% win rate on 2.6 bets is not statistically distinguishable from a coin flip with a good payoff.
It is gross of costs. The realised-PnL code weights returns but never subtracts commission or slippage.

4. Is there hindsight bias?

I audited the feature construction, the walk-forward harness, the live training path, the universe, and the reporting. The verdict is nuanced: no look-ahead in the algorithm, but material bias in the reporting and in the selection of what gets reported.

Check	Finding	Severity
Feature lookahead	No negative shifts, no centred windows, no back-fill. Every feature is a trailing rolling/EWM statistic. Clean.	none
Walk-forward protocol	Expanding window, scaler fitted inside each fold, baskets originated every 5 trading days (non-overlapping). Clean.	none
Label embargo	Training cut-off is `d − 5 calendar days`, but the label is a 5-trading-day return (~7 calendar days). The last ~2 days of training labels overlap the test window.	minor
Live model training	`ml_model.py` fits the full 5-year sample with a random 15–20% validation split for early stopping. In-sample optimism — but this model is not what the walk-forward evaluates.	moderate
Universe	A fixed list of 50 current EU/Scandi large caps. No delisted names, no point-in-time constituents — survivorship and selection bias.	moderate
Variant selection	The short leg was dropped because the walk-forward showed it losing money, and the headline arm is the no-volume variant. Both choices are defensible, but they mean the long-only walk-forward is a post-selection estimate, not a clean out-of-sample test of the traded strategy.	moderate
Macro features	FRED series are revised (final values, not vintage). Yield-curve and VIX levels dominate importance, so the model can memorise the 2024–26 regime.	moderate
Reporting	The headline compounds 13 overlapping 5-day windows as if sequential, and the realised PnL is gross of costs. This is the largest distortion on the page.	high
Data quality	The volume series has 2 NaN PnLs and 5 duplicated origin dates with conflicting values (92 rows vs 90 in the Kelly file).	moderate

Verdict. The feature pipeline is clean — every feature is a trailing statistic, the target is a genuine forward return, and the walk-forward refits the scaler inside each fold on data that ends before the test date. There is no smoking-gun look-ahead. The bias that matters is survivorship (a fixed list of today's 50 names), selection (the short leg and the headline arm were chosen after seeing the walk-forward), and above all reporting (overlapping-window compounding, gross of costs, n_eff omitted).

5. What to fix, in priority order

Report a capital-base return. Replace the cumulative product of overlapping windows with a cohort simulation (the same trades are worth ~+9% here, not +52.6%).
Deduct costs in the realised PnL. The live series is gross; at 20 bps/leg the walk-forward Sharpe falls from 2.04 to 1.34 and annualised return from 32.8% to 20.1%.
Fix the volume series. Drop the 2 NaN rows and de-duplicate the 5 conflicting origin dates before any cumulative figure is computed from it.
Lengthen the label embargo to 7–8 calendar days (or drop the last 5 trading rows per ticker) so training labels cannot overlap the test window.
Hold out a true out-of-time set for the live model and use time-ordered early stopping instead of a random validation split.
Test survivorship. Rebuild the universe point-in-time, including names that were delisted or acquired, and re-run the walk-forward.
Publish n_eff next to every win rate. A 76.9% hit rate on 2.6 independent bets is not evidence of a 76.9% edge.
Decide whether beta 1.06 is intended. Half the long-only return is the market; if the sleeve is meant to be alpha, it needs hedging or beta-aware sizing.

6. Data quality note

The volume A/B series is not currently usable as a performance record: it contains 2 NaN PnLs and 5 duplicated origin dates with conflicting values (92 rows, versus 90 in the Kelly file). De-overlapping its 85 unique windows turns the reported +27.5% into -6.8% — the volume arm has not been additive on a capital-base basis. Fix the series before quoting it.

Method & provenance

Walk-forward: walkforward_eval.py — expanding window, training cut-off d − 5 calendar days, scaler fitted per fold, baskets originated every 5 trading days, top/bottom 10 of 50, costs applied per leg. Panel: 8,842 scored rows, 91 signal dates, 2 Sep 2024 → 8 Jun 2026. Live PnL: data/ml_realized_5d_performance.csv (13 windows, 14 Oct → 3 Nov 2026) and ..._vol.csv. Sizing: data/kelly_sizing.json. Feature importance: permutation on sklearn_gb_reg_volpruned_model.pkl, last 20% of dates, 5 repeats. All figures recomputed 4 November 2026; the walk-forward cache ends 8 Jun 2026, so it does not cover the Oct–Nov live run.

Not investment advice. Past performance — especially 27 days of it — is not indicative of future results.