Adoption Analysis Data Packages
Each adoption analysis data package is a zip archive that contains results and underlying data for one app. Each contains 4 subfolders: plots, raw, results, and weights.
These files reside in the results subfolder and provide everything you need to evaluate the adoption phase’s impact on performance metrics. All other files in the data package contain either underlying data you can use to validate our impact analysis methodology or different views of the same data.
summary.csv - The summary.csv file in the results subfolder summarizes the results of the adoption phase. The columns are:
avg_impact - Average daily impact. Positive values are relative increases in the metric.
avg_impact_pct - Average daily impact as % difference between actual values of the metric and the synthetic control. Positive values are relative increases in the metric.
total_impact - Total impact. Positive values are total increases in the metric across the entire adoption phase. Not applicable for percentage metrics like ROAS (you can't total percentages).
total_impact_pct - Total impact as a % difference between the sum of actual values and sum of predicted synthetic control values during the adoption phase. This similar to avg_impact_pct, but uses totals instead of averaging the daily % differences. Not applicable for percentage metrics like ROAS (you can't total percentages).
all.csv - The all.csv file in the results subfolder will usually be the only file you need if you want more details than summary.csv can provide. You can filter by country, metric, and "xday" (cohort day) to see a single set of daily results. The columns are:
actual - The actual value of the metric in the adoption country.
prediction - The synthetic control's predicted value of the metric in the adoption country.
rel_impact - The relative impact of the adoption phase on that individual date (actual minus prediction). Positive values are increases. Only applies to post-treatment dates (after the adoption phase started).
cum_impact - The cumulative impact over the course of the adoption phase up to that date. Positive values are cumulative increases. Only applies to post-treatment dates (after the adoption phase started).
results - The primary results of the adoption phase analysis. These files contain everything you'll really need. Please see the Aggregated Results section above for more information on the files in this subfolder.
plots - Graphical plots of the actual metrics for each country that participated in the adoption phase and the synthetic controls. Very similar in nature to the plots we display in the Experiment Results page. The top graph plots actual metrics vs the synthetic control. The middle graph plots the differences (positive values after the vertical dotted line in this graph are relative increases in the metric on each date due to the adoption phase). The bottom graph plots the cumulative differences (positive values after the vertical dotted line in this graph are cumulative increases in the metric over the course of the adoption phase).
weights - The weights used in each synthetic control. Weights are multiplied by each indicated country level metric and then summed to get the "regression" component.
components - The components used in each synthetic control. Components are additive so sum them to get the prediction.
raw - The raw data behind the plots. Can be ignored unless you want to see the underlying data (which is mostly duplicated in the results files anyway).
Aggregated Results
These files reside in the results subfolder and provide everything you need to evaluate the adoption phase’s impact on performance metrics. All other files in the data package contain either underlying data you can use to validate our impact analysis methodology or different views of the same data.
summary.csv - The summary.csv file in the results subfolder summarizes the results of the adoption phase. The columns are:
avg_impact - Average daily impact. Positive values are relative increases in the metric.
avg_impact_pct - Average daily impact as % difference between actual values of the metric and the synthetic control. Positive values are relative increases in the metric.
total_impact - Total impact. Positive values are total increases in the metric across the entire adoption phase. Not applicable for percentage metrics like ROAS (you can't total percentages).
total_impact_pct - Total impact as a % difference between the sum of actual values and sum of predicted synthetic control values during the adoption phase. This similar to avg_impact_pct, but uses totals instead of averaging the daily % differences. Not applicable for percentage metrics like ROAS (you can't total percentages).
all.csv - The all.csv file in the results subfolder will usually be the only file you need if you want more details than summary.csv can provide. You can filter by country, metric, and "xday" (cohort day) to see a single set of daily results. The columns are:
actual - The actual value of the metric in the adoption country.
prediction - The synthetic control's predicted value of the metric in the adoption country.
rel_impact - The relative impact of the adoption phase on that individual date (actual minus prediction). Positive values are increases. Only applies to post-treatment dates (after the adoption phase started).
cum_impact - The cumulative impact over the course of the adoption phase up to that date. Positive values are cumulative increases. Only applies to post-treatment dates (after the adoption phase started).
Subfolders
results - The primary results of the adoption phase analysis. These files contain everything you'll really need. Please see the Aggregated Results section above for more information on the files in this subfolder.
plots - Graphical plots of the actual metrics for each country that participated in the adoption phase and the synthetic controls. Very similar in nature to the plots we display in the Experiment Results page. The top graph plots actual metrics vs the synthetic control. The middle graph plots the differences (positive values after the vertical dotted line in this graph are relative increases in the metric on each date due to the adoption phase). The bottom graph plots the cumulative differences (positive values after the vertical dotted line in this graph are cumulative increases in the metric over the course of the adoption phase).
weights - The weights used in each synthetic control. Weights are multiplied by each indicated country level metric and then summed to get the "regression" component.
components - The components used in each synthetic control. Components are additive so sum them to get the prediction.
raw - The raw data behind the plots. Can be ignored unless you want to see the underlying data (which is mostly duplicated in the results files anyway).
Updated on: 04/01/2023
Thank you!