Future

Research opportunities, completed experiments, and the path to production

Unfinished Experiments

Work in progress from Phases 7–10 that needs completion

experiment	effort	potential	status
Phase 7B: Regime-Aware Trading	Med	High	Complete
Implemented spread-based regime detection on z-score series. Regimes: Ranging (0), Trending Up (1), Trending Down (2), High Volatility (3). Filters skip entries during unfavorable regimes (e.g., skip short entries when z-score trending up). Target: Sharpe +0.5, Win Rate +3pp on META pairs. See results page.
Phase 7C: Risk Management	Low	Medium	Not started
Add stop-losses and take-profits to winning AMZN configuration. Could lock in gains from +1.32% winning strategy.
Phase 9: Pairs Normalization + Exit Sweep	Med	High	Complete
Completed. Per-pair normalization implemented in training and backtest. Exit z-score swept [0.1–1.5] per pair. Model scaled to 4.3M params (emb 512). Final results: META-TSLA +31.4%, Sharpe 6.51; META-MSFT +24.1%, Sharpe 5.72; AMZN-META +5.8%. AAPL pairs still 21–28% win rate despite fix. See results page.
AAPL Spread Normalization	Low	High	Complete
Fixed. Replaced global mean/std with per-pair normalization in both `src/log_return_data_pairs.py` (training) and `scripts/backtest_pairs_model.py` (inference). AAPL win rate improved slightly (16–21% → 21–28%) but gap persists — this is now a learning problem, not a normalization problem.
Phase 9 Exit Strategy Sweep	Low	High	Complete
Completed. Swept exit z-score per pair via `scripts/sweep_exit_params.py`. Best params saved to `results/phase8_exit_sweep/best_per_pair_params.json`. `exit_zscore = 1.5` optimal for 7/10 pairs; model-confidence exit mode was uniformly worse (all pairs unprofitable).
Phase 9C: Per-Pair Fine-Tuning (AAPL)	Med	High	Complete
Completed. Per-pair adapter MLPs added (512→128→512, residual). Two-phase training: backbone warmup then full fine-tune. Key discovery: adapters balanced NLLs (all pairs −4.995 ± 0.004) but AAPL pairs remain catastrophic (16–19% win rate). The mean-reversion z-score strategy is fundamentally incompatible with AAPL's microstructure — not a model quality issue.
Phase 10D: Momentum Pair Strategy	Low	High	Complete — negative result
Completed. Swept 48 param combos across 6 pairs. Momentum is uniformly worse than mean-reversion across all pairs — including the near-breakeven AMZN-MSFT (−2.09 Sharpe delta) and MSFT-TSLA (−2.59 delta). The H≈0.89 in the OLS spread level is not a tradeable momentum signal; the mean-reversion z-score trades the deviation from equilibrium, not the raw spread direction. AMZN-MSFT and MSFT-TSLA are structurally marginal and excluded from the portfolio.

Phase 10: Portfolio Combinator — Results

Completed June 2026 — combining the 3 profitable pairs into a shared-capital portfolio

mode	portfolio return	sharpe	max drawdown	vs best solo pair
Equal-Weight	+129.3%	25.21	1.2%	+4.4× Sharpe vs META-TSLA solo
Dynamic Kelly	+130.2%	25.74	1.2%	+2.1% Sharpe improvement over equal-weight

The key finding: all three pairs have near-zero realized P&L correlation (ρ < 0.06), despite META appearing in all three pairs. The z-score entry gate staggers trade timing — losses don't compound. Portfolio Sharpe of 25+ emerges purely from diversification, not from any change to the underlying strategy.

Dynamic Kelly correctly allocates less to META-TSLA (lowest individual Sharpe 13.0) and more to META-MSFT (15.8) and AMZN-META (16.2), converging toward the highest-quality signal sources over time.

Research Directions

New experiments based on learnings from 17+ completed runs

Ensemble Methods V2

Phase 7D showed ensemble of checkpoints produces identical results (models too correlated). Try training truly independent models:

Different architectures (Conv1D + LSTM + Transformer ensemble)
Different timeframes (1-min + 5-min + 15-minute aggregation)
Different feature subsets (technical-only vs price-only vs volume-only)

Hypothesis: Independent errors will cancel, producing more robust predictions.

Alternative Prediction Targets

Currently predicting log-returns. Test other targets:

Return direction — Binary classification (up/down) with confidence
Volatility — Predict next-tick variance for sizing decisions
Price level — Direct price prediction (not log-return)
Rank — Which stock in universe will perform best

Attention Visualization

Transformer encoder has attention weights. Visualize what the model "looks at" in the 30-tick window:

# Extract attention weights during inference
attn_weights = model.encoder.layers[-1].attention_weights
# Shape: [batch, heads, seq_len, seq_len] — shows which ticks matter

Goal: Understand if model focuses on recent ticks, volatility spikes, or specific patterns.

Market Regime Transfer

Winning configuration only works on AMZN. Test hypothesis: Does AMZN model work on other stocks during similar market conditions?

Cluster historical regimes across all stocks
Test AMZN model on META when META is in "AMZN-like" regime
Potentially explains stock-specific pattern learning

Engineering

Infrastructure and scaling work

project	effort	description
Live Trading Infrastructure	High	Real-time data feed (WebSocket), order execution, paper trading mode
Expanded Universe	Med	Add 10-20 more stocks, test cross-stock generalization
Model Compression	Low	Quantization (INT8), pruning for edge deployment
Auto-Hyperparameter	Med	Bayesian optimization for Kelly fraction, min return, thresholds
Multi-GPU Training	Med	Distributed training for faster iteration on large experiments

Production Path

Operationalizing the research for real-world deployment

Model Monitoring

Drift detection — Alert when feature distributions shift
Performance dashboards — Real-time win rate, Sharpe, drawdown
Retraining triggers — Auto-retrain when performance degrades
A/B testing — Shadow mode: new model vs production model

Risk Framework

Portfolio-level Kelly — Correlation-aware position sizing across multiple positions
Drawdown limits — Circuit breakers at -5%, -10% portfolio level
Exposure limits — Max sector allocation, max single-stock risk
Regime hedging — Auto-hedge or reduce size in detected bear regimes

Explainability

SHAP values — Feature importance for each prediction
Counterfactuals — "What would need to change for opposite signal?"
Prediction logs — Full traceability: input → embedding → output

Regulatory & Compliance

Audit trails — Every prediction, decision, and trade logged
Compliance reporting — Regulatory filings (if institutional)
Model cards — Documentation of training data, performance, limitations

Priority Matrix

priority	items	rationale
Immediate	Expanded universe — add 10+ new stocks, mine new uncorrelated pairs	Portfolio Sharpe scales with number of uncorrelated pairs; Phase 10 proved near-zero P&L correlation even with shared stocks
Near-term	Expanded universe (add 10+ stocks), attention visualization, run regime sweep validation	More pairs = more diversification; validate Phase 7B regime filter targets (+0.5 Sharpe, +3pp win rate)
Medium-term	Live trading infra, model monitoring dashboards, retraining triggers	Portfolio Sharpe 25+ warrants production deployment exploration
Long-term	Risk framework (circuit breakers, regime hedging), regulatory compliance, explainability	Institutional-grade operation

Key Questions to Answer

Does pairs trading work? — Yes. Portfolio of 3 pairs: Sharpe 25.21, MaxDD 1.2%, +129% return (Phase 10B).
Why do AAPL pairs fail? — Answered. Mean-reversion strategy is fundamentally incompatible with AAPL microstructure (Phase 9C). Excluded from portfolio.
Does combining pairs improve risk-adjusted returns? — Dramatically. Portfolio Sharpe 25 vs best solo Sharpe 6.5 — near-zero P&L correlation between pairs despite shared META exposure.
Does dynamic Kelly beat equal-weight? — Modestly. +2.1% Sharpe improvement. Dynamic Kelly converges toward META-MSFT + AMZN-META (higher individual Sharpe).
Can regime detection improve returns? — Implemented. Phase 7B complete: spread-based regime detection skips entries during trending regimes. Target: +0.5 Sharpe, +3pp win rate on META pairs.
What's the live Sharpe? — Backtest portfolio Sharpe 25+; live will differ. Next step: paper trading mode.
Can momentum strategy rescue near-breakeven pairs? — No. Momentum uniformly worse across all 6 pairs (Phase 10D). High H in OLS spread is a construction artefact, not a signal.