> ## Knowledge Base Index
> Fetch the complete knowledge base index at: https://help.metric.works/sitemap.xml
> Use this file to discover available pages before exploring further.
> Pure-Markdown content can be obtained by appending a '.md' suffix to the content URLs listed in the sitemap (without the trailing slash).

# Backtesting Error

# Basics

Model error refers to the difference between backtesting predictions of the model and the actual observed values. It is meant to provide some insight into **how well the model is able to predict a future holdout test set with its current fit**.

This is not a comprehensive evaluation of the model’s accuracy and cannot be used to validate a media mix model (MMM). **Validation is only possible with ground truth incrementality, which is why experiments are so important**. In fact, backtested normalized root mean squared error (NRMSE) is only one of several optimization objectives utilized in the training process. Model error can only give an important, but limited view of a model’s fit and correlative prediction power.

---

# Errors File

The predicted column contains the backtesting predictions. The actual column contains the observed values. Ideally, those numbers are close. If they aren't, it could mean there are [unmodeled performance factors](/en/article/incrementality-analysis-mcgt93/#3-unmodeled-performance-factors). The abs_pct_error column contains the [mean absolute percentage error](https://en.wikipedia.org/wiki/Mean_absolute_percentage_error) or MAPE, which is another way to look at the difference.