Structural Diagnostics

Overview

This page documents diagnostics for structural misspecification checks after fitting:

• posterior pair plots for geometry and parameter dependence,
• residual-versus-predictor plots for nonlinearity detection.

Both write outputs to 50_diagnostics/.

1) Pair Plot

Function Signature

from src.diagnostics.pair_plot import generate_pair_plot

generate_pair_plot(
    model: Any,
    config: dict[str, Any],
    results_dir: str,
) -> str | None

Artefact

Filename	Stage folder	Description
pair_plot.png	50_diagnostics/	Pairwise scatter plot for available variables among beta_channel, alpha, lam, with divergence markers when available.

Behaviour

• Requires at least two target posterior variables to plot.
• Uses divergences=True only when idata.sample_stats["diverging"] exists.
• Returns saved path on success, else None.

2) Residuals vs Predictors

Function Signature

from src.diagnostics.residuals_vs_predictors import generate_residuals_vs_predictors

generate_residuals_vs_predictors(
    model: Any,
    X_train: pd.DataFrame,
    y_train: pd.Series | np.ndarray,
    config: dict[str, Any],
    results_dir: str,
) -> list[str]

Artefacts

Filename pattern	Stage folder	Description
residuals_vs_{channel}.png	50_diagnostics/	One plot per media spend column from config["media"][*]["spend_col"], with residual scatter and LOWESS trend (if statsmodels is available).

Behaviour

• Computes residuals as observed - median posterior predictive.
• Attempts inverse-transform if target scaling is active.
• Returns list of saved paths (possibly empty).

Interpretation Guidance

Diagnostic	Signal	Typical interpretation
Pair plot with divergence concentration	Sampler geometry issue	Potential funnel/identifiability problem; revisit parameterisation and priors.
Strong curved LOWESS in residuals vs spend	Missed nonlinearity	Saturation/adstock specification may be inadequate for that channel.
Residual trend around non-zero mean	Systematic bias	Revisit controls, trend/seasonality design, or likelihood assumptions.

Usage Example

from src.diagnostics.pair_plot import generate_pair_plot
from src.diagnostics.residuals_vs_predictors import generate_residuals_vs_predictors

generate_pair_plot(driver.model, driver.config, driver.results_dir)
paths = generate_residuals_vs_predictors(
    driver.model,
    driver.X_train,
    driver.y_train,
    driver.config,
    driver.results_dir,
)
print(f"Generated {len(paths)} residual plots")

Relationship to Workflow Stages and Gates

• Stage: 50_diagnostics/.
• These are structural diagnostics that complement hard machine gates.
• They support analyst gate review and model revision decisions when residual shape or posterior geometry indicates misspecification.