Guide: Prior Calibration from Historical Runs

This guide explains how to derive channel-prior suggestions from historical AMMM results.

What This Guide Covers

• Input requirements for the calibration utility
• Current V2 file paths to use
• Practical usage and caveats

Tool

Utility script:

python research/prior_calibration/priors_calibration.py --help

Required Inputs

The utility expects one or more run result directories containing:

• 20_model_fit/model_summary.csv

It extracts rows where parameter starts with beta_channel[...].

Typical Usage

python research/prior_calibration/priors_calibration.py \
  --results-dirs runs/2025-08-01/results runs/2025-09-12/results \
  --config-yaml data-config/demo_config.yml \
  --dist HalfNormal \
  --default-sigma 2.0 \
  --out suggested_priors.yml \
  --verbose

Key flags:

• --results-dirs: one or more AMMM results roots.
• --config-yaml: source of canonical media-channel order.
• --dist: HalfNormal (default) or Normal.
• --default-sigma: fallback scale.
• --out: optional output YAML path.

Output

The script prints (and optionally writes) a YAML block suitable for custom_priors.beta_channel.

Validation Checks

1. Ensure all source runs are methodologically comparable.
2. Confirm channel ordering matches current media[].spend_col.
3. Run prior predictive checks after applying new priors:
   • 10_pre_diagnostics/prior_predictive_check.png
   • 10_pre_diagnostics/prior_predictive_summary.csv

Caveats

• Historical posteriors can encode past bias and misspecification.
• Prior calibration improves plausibility but does not guarantee causal validity.
• Always compare calibrated vs baseline priors with the same dataset.