Backtesting vs Forward Testing: Differences and Benefits

In trading, backtesting and forward testing are essential methods for evaluating the potential success of trading strategies.

Backtesting allows traders to assess how a strategy would have performed in the past by simulating trades with historical data. Unlike backtesting, forward testing involves trading a strategy in real-time with live data in a demo account, without risking real money.

These often seem like optional steps to newer traders, or an either/or scenario.

But in this article, I’ll show you why they are both necessary in the development of trading strategies and cannot be skipped.

Defining Backtesting and Forward Testing

Backtesting

This is a technique used to test your trading strategy using historical data.

The image above is an example of a backtest that I did recently.

In essence, you are seeing how your strategy would have performed in the past.

By analyzing historical market data, you can identify patterns and assess the potential profitability of your strategy without risking real money.

What worked in the past is generally likely to work in the future.

A perfect example of a successful hedge fund that has done extensive backtesting is Renaissance Technologies.

Their unique mathematical models and extensive backtesting have made them extremely successful.

Forward Testing

Also known as paper trading this involves testing your strategy in real-time, with live data.

However, instead of committing real money, you simulate trades to predict how your strategy performs in current market conditions.

This method allows you to assess the practicality and adaptability of your strategy, when faced with market dynamics and volatility.

You’ll also find out if your strategies fit your personality and schedule.

Here’s a quick comparison of the 2 methods:

Both backtesting and forward testing play crucial roles in the development of a successful trading strategy.

Through backtesting, you gain a historical understanding of your strategy’s performance, while forward testing offers a real-time perspective.

It’s important to go through both processes to ensure that you’ve been thorough in your testing. 

Purpose and Goals

Backtesting and forward testing each serve a specific function in the trading strategy development and validation process.

Here’s a more detailed look at each one and what they will help you with.

Purpose of Backtesting

Backtesting is a method used by traders to evaluate the potential of a trading strategy by applying it to historical data.

This process helps in understanding how a strategy would have fared in the past, allowing for adjustments and optimization before deploying it in live markets.

By identifying potential weaknesses and strengths, backtesting reduces the risk of future losses and improves the likelihood of success.

It also enables the testing of various parameters, conditions and ideas to fine-tune strategies for maximum effectiveness.

Furthermore, backtesting provides insights into the risk and return profile of a strategy, helping in managing expectations and investment decisions.

Through this method, traders can gain confidence in their strategies, ensuring they are well-prepared for a wide range of market conditions.

Purpose of Forward Testing

Forward testing, also known as paper trading, involves applying a trading strategy in real-time markets without using actual capital.

This technique allows traders to evaluate a strategy’s performance under current market conditions, offering insights into its practical viability and effectiveness.

Forward testing helps in identifying any unforeseen flaws or areas for improvement in a strategy that might not have been apparent during backtesting.

It bridges the gap between theoretical backtesting results and actual trading, providing a more realistic assessment of how a strategy performs.

This method also enables traders to familiarize themselves with the strategy’s mechanics in a live market environment.

For example, your strategy may have been very profitable in backtesting, but you may discover in forward testing that you don’t have the time to manage the trades.

If that is the case, then you might find that you have to go back to the drawing board and create a strategy on a longer term chart.

By mitigating the risk of significant losses through virtual trading, forward testing is an essential step in validating and fine-tuning a trading strategy before committing real capital.

Methodologies

In trading system evaluation, proven procedures for backtesting and forward testing are essential for getting usable data.

Backtesting Process

Here are the steps that are required to do a backtest.

For a more detailed description, read the complete guide.

Before you implement these steps, be sure that you have already selected a market, timeframe and trading strategy that you want to test.

1. Choose a software platform and download historical data: Begin by choosing a backtesting platform and downloading historical market data, which includes prices, volumes, and other relevant information.

This is available on most backtesting platforms.

You want your backtesting data to go as far back as possible.

2. Strategy coding: For an automated strategy, encode your trading strategy into a software application that can execute the strategy.

If you’re using a discretionary strategy, create a written trading plan.

3. Backtest: Run the strategy against the historical data to simulate trading results.

4. Analysis: Review the results.

Remember that there are no “perfect” results.

You have to determine what your goals are and if a strategy meets your goals.

Here are key metrics to focus on:

5. Optimization: After the initial simulation, you may find that your results do not meet your goals.

This is common.

If that’s the case, optimize your strategy by adjusting the parameters and retesting.

6. Validation: Once you have a strategy that you’re satisfied with, validate the strategy by applying it to out-of-sample data.

Repeat this process as many times as necessary until you have a trading strategy that you’re satisfied with.

After your strategy passes the steps above, you’re not done yet.

This is where most new traders stop.

But not you.

Now it’s time to go through the forward testing process.

Forward Testing Process

1. Setup a demo/paper trading account: Create a demo account to simulate transactions using real-time data, without committing real capital. Many brokers and trading platforms have this option available at no cost. You can also use a simple notebook or spreadsheet to record your trades.

2. Account sizing: Select an account size that will be similar to the amount of risk capital that you’ll use once you have a strategy that’s completely tested.

3. Execute trades: Setup your charts like you did in your backtesting and start taking demo trades.

Use the same code for an automated strategy or your trading plan for a discretionary strategy.

4. Analysis: Review the results and see if they meet your goals.

It will probably take some time to compile enough meaningful data, so be patient.

5. Iteration: Based on the live performance data, make tweaks to the strategy and revert to backtesting to check these adjustments.

6. Expansion: If the strategy shows similar results to backtesting, you may consider starting to trade it with real money.

It’s generally a good idea to increase the size of a live account gradually, while maintaining the same risk management.

It may be beneficial to start backtesting the strategy in other markets and on other timeframes at the same time.

Repeat the process of backtest, optimize, forward test, optimize, until you have enough strategies to meet your income goals.

Advantages and Disadvantages

When you consider using backtesting and forward testing for your trading strategies, it’s important to understand the unique benefits and potential drawbacks of each one.

They are complementary, so while they do have overlapping benefits, they test entirely different things.

Pros and Cons of Backtesting

Pros:

• Quick Results: You can conduct backtesting relatively quickly because it uses historical data. Manual backtesting can be slow, but it’s significantly faster than learning in real-time. Automated backtesting is very fast and can give you results in just a few minutes.
• Cost-Efficient: No real money is at risk while testing historical scenarios. Backtesting software is also very affordable and some solutions are even free.
• Confidence Building: A successful backtest will give you the initial level of confidence that your strategy works. If you don’t have a minimum level of confidence, you’ll always second guess yourself in live trading because you have no proof that your strategy has an edge.

Cons:

• Overfitting Risk: Backtesting can lead to strategies that are overly optimized for past data but may not perform well in future markets.
• Human Error: It’s possible to make mistakes when backtesting. When testing an automated strategy, there can be errors in the coding or logic of the strategy. In discretionary backtesting, it’s possible to make errors in interpreting the rules or changing the rules in the middle of a test.  Not accounting for normal trading fees can also lead to unrealistic results.
• Not Real Time: Since you’re not trading in real-time, it won’t factor in time stress.

Strengths and Weaknesses of Forward Testing

Strengths:

• Real Market Conditions: Forward testing your strategy exposes it to current market conditions, which are not available with historical data.
• Psychological Preparedness: You get a better sense of how you’ll react emotionally to real-time market movements.
• Cost-Efficient: Many brokers and trading platforms offer free demo accounts for you to practice. Since no money is on the line, you’re free to make mistakes without losing money.

Weaknesses:

• Time-Consuming: It can require a substantial amount of time to gather enough data for analysis.
• Loss of Focus: Since trades don’t setup as frequently as in backtesting, it can be easy to lose focus.
• Doesn’t Simulate Stress of Loss: Since real money is not on the line, the psychology is a little different from real-money trading. If you want to better simulate real trading conditions, consider forward testing in a very small live account.

Backtesting vs Forward Testing: Which One is Better?

Both trading backtesting and forward testing serve critical but different roles in strategy development.

This does not make one necessarily better than the other.

They are complementary.

Backtesting provides a first insight into a strategy’s historical performance, allowing for rapid iterations and adjustments, without financial risk.

It helps identify potential strengths and weaknesses over a wide range of market conditions in the past.

However, it might not account for all real-world variables, such as liquidity issues or slippage, leading to potentially over-optimistic results.

Forward testing, on the other hand, offers a more realistic view of how a strategy performs under current market conditions and can highlight issues not apparent in backtesting.

While it’s more time-consuming and requires patience, it helps validate the real-world performance of a strategy.

Ultimately, the most effective approach combines both methods, using backtesting for initial strategy development and refinement, followed by forward testing to confirm its real-world viability.

What is the Difference Between Backtesting and Out-of-Sample Testing?

In a nutshell, out-of-sample testing is a subset of backtesting and it used to validate the backtesting results with historical data that was not used in the original backtest.  

Using out-of-sample testing and backtesting are both methods used to evaluate trading strategies, but they differ in the data they utilize.

Backtesting involves running a strategy against historical data to assess its performance.

In contrast, out-of-sample testing evaluates the strategy’s effectiveness on a separate set of data not used during the development phase, offering a more unbiased measure of its real-world applicability.

This can be accomplished by using only part of the available historical data for backtesting and optimization. Once a strategy works well in backtesting, it can be further backtested on the rest of the data that was not used in the initial backtesting and optimization process. 

While backtesting helps refine and optimize a strategy, out-of-sample testing provides a crucial check against overfitting, ensuring the strategy can perform well under previously unseen market conditions.

Again, both methods are complementary, with backtesting focusing on strategy development and optimization, and out-of-sample testing emphasizing validation.

Conclusion

So that’s the difference between backtesting and forward testing and why it’s important to do both.

Many aspiring traders skip both of these steps and that’s why over 90% of traders fail.

When you do both steps, you’ll have a very high level of confidence that your strategies work and will be less likely to hesitate when taking trades.

To get started, be sure to read my backtesting and forward testing guides.