Methodology
In-Sample vs Out-of-Sample Backtesting: Crypto Guide
A practical guide to holdout periods, walk-forward testing, and running the same finalized Traseq strategy across separate crypto backtest windows.
Traseq··7 min read
A practical guide to holdout periods, walk-forward testing, and running the same finalized Traseq strategy across separate crypto backtest windows.
Start with a no-code crypto spot strategy, lock the version, run the backtest, and keep the result traceable for comparison.
In-sample vs out-of-sample backtesting is a simple way to check whether a crypto strategy only fits the data you used while building it. You tune rules on one historical window, freeze the rules, then test the same rules on a later window you deliberately held back.
Traseq supports this as a research workflow: build the strategy, finalize the version, run separate backtests across different date ranges, and compare the results. Traseq is not live trading, exchange execution, a broker terminal, or a guarantee of future performance.
Use the in-sample period to build and tune the strategy. Use the out-of-sample period only after the rules are frozen. If the same finalized logic behaves reasonably across both windows, the result is stronger evidence than an in-sample-only backtest; if it collapses on the holdout, the in-sample result was probably overfit.
The key rule is strict: once you look at the out-of-sample result, do not tune the same strategy to fix it. That would turn the holdout into another in-sample window.
The terms describe how a slice of historical data was used during research:
This matters because every parameter choice is a form of learning from history. If you grade the strategy on the same window that shaped those choices, the grade is inflated. Out-of-sample testing creates a cleaner test because the rules were not designed around that window.
The failure mode is overfitting in backtesting: the strategy describes one historical slice instead of behavior that can survive a different slice.
Suppose you have two years of BTCUSDT 1d data from 2024-01-01 to 2025-12-31. A practical first split could look like this:
This is not the only valid split. A shorter strategy might need a longer history to collect enough trades. A slower strategy might need wider windows so each period includes more than one market condition. The principle stays the same: pick the holdout before you look at the answer.
You do not need a dedicated split button. Use Traseq's version-linked backtest workflow:
Because both runs point to the same finalized strategy version, you can trace the result back to identical logic. That is the main reason versioning matters for this workflow. For the underlying model, see Core Concepts; for mechanics, see Run Your First Backtest.
Do not judge the split by return alone. Compare the behavior:
Some degradation is normal. The warning sign is not "out-of-sample is worse." The warning sign is a result that only worked in the window you tuned against. Use how to compare backtest results and the compare backtests guide when you put the runs side by side.
A single IS/OOS split gives you one holdout verdict. That is useful, but it still depends on the windows you chose. Walk-forward testing repeats the same idea across rolling windows:
In Traseq, you can approximate this manually by finalizing a version, running it across several successive date ranges, and comparing all runs in one comparison set. It is more manual than a dedicated optimizer, but the research discipline is the same: freeze the logic before testing a window that did not shape the rules.
Out-of-sample testing improves research quality, but it is not proof of future behavior.
Treat out-of-sample results as stronger evidence, not certainty. The right next step is usually comparison: test the same finalized logic across more windows, then compare what changed and what stayed stable.
Ready to try the split yourself? Run the interactive demo, then build a strategy version and compare an in-sample run against an out-of-sample run in Traseq.
In-sample backtesting uses the data you looked at while building and tuning the strategy. Out-of-sample backtesting uses a separate holdout period after the rules are frozen, so the strategy was not designed around that specific window.
It grades the strategy on the same history that influenced the rules. That can make a fragile strategy look stronger than it is, especially when many parameters, filters, or date ranges were tried during development.
Pick it before you tune the strategy. The holdout should be late enough to be meaningfully separate, long enough to produce useful trade data, and representative enough that it is not just one narrow market condition.
Tune the strategy on the in-sample date range, finalize the version, then run the same finalized version on the out-of-sample date range. Keep the symbol, timeframe, fees, slippage, capital, and fill assumptions the same, then compare both runs in a comparison set.
Walk-forward testing repeats the in-sample/out-of-sample split through time. You tune on one window, validate on the next, move both windows forward, and repeat so the conclusion does not depend on one holdout period.
No. It is stronger research evidence than an in-sample-only result, but it is still based on historical data. It does not guarantee future performance, remove market-regime risk, or turn a backtest into a trading recommendation.
| Period | Date range | Role | What to do |
|---|
| In-sample | 2024-01-01 to 2025-06-30 | Build and tune | Try rule variants, adjust parameters, remove obvious mistakes |
| Out-of-sample | 2025-07-01 to 2025-12-31 | Validate only | Run the frozen version; do not adjust rules to improve this result |