Causal Structure
Causal Structure Recovery under Partial Observability
The standard estimate-then-discover pipeline for causal inference in partially observed dynamical systems is systematically biased when built on RTS-smoothed states. Filtered state estimates preserve the exogeneity conditions required for Granger-style discovery; switching representation recovers near-oracle causal structure at no extra cost.
- Causal inference
- State-space
- Time series
- Granger
- Kalman
In real-world settings (power systems, sensor networks, neural recordings) the variables whose causal relationships we want are hidden; we observe noisy projections. The standard two-step pipeline: (1) recover the hidden trajectory with a Kalman filter or RTS smoother; (2) run causal discovery (Granger, partial correlation, VAR) on it.
The paper argues step 1 is not neutral — the choice of state representation determines whether step 2 is valid. (A filter uses only past observations; a smoother uses the whole trajectory including the future. Smoothers are lower-variance — that’s the trap.)
Structural, not empirical
RTS smoothing pulls future information into the estimate at every timestep — x̂ₜ|T = f(y₁,…,yₜ,…,y_T). Regressing this on its own past for Granger causality correlates the regressor with the error term; OLS exogeneity fails → biased.
The filtered estimate x̂ₜ|ₜ = f(y₁,…,yₜ) depends only on the past, preserving exogeneity → the same test returns near-oracle structure. The error lives at the pipeline level, not in the method — changing the discovery algorithm won’t fix it; changing what you feed it will.
Three-node ground truth
A three-node ground-truth graph over x, y, z (two edges: x→y, y→z) passed through a partially observed linear dynamical system; switch the state representation (True / Filtered / Smoothed / Innovations) and watch which edges the Granger test finds.
Linear-Gaussian state-space system with a known two-edge causal graph + a partial observation matrix; each of four representations fed into Granger F, VAR coefficient tests, partial correlation, and LASSO. Sweeps: observability (rank 1→full), sample size (200→20,000 timesteps), process/observation noise, system dimension (3→12 latents). ~2,400 synthetic systems.
Which causal structure each state representation recovers.
| Representation | Signal | Bias | Discovered |
|---|---|---|---|
| True | preserved | none | oracle |
| Filtered | preserved | negligible | near-oracle |
| Smoothed | distorted | structural | false edges |
| Innovations | removed | n/a | no edges |
Critically, the smoothed-state false-positive rate increases with sample size — ruling out finite-sample noise; it’s a structural artefact of the representation. All four discovery methods agree.
In any estimate-then-discover pipeline on a partially observed system, use filtered states instead of smoothed states (rts_smooth() → kalman_filter()) — a one-line swap; the Kalman recursion already produces the filtered estimate.
Limits: assumes linear-Gaussian systems with correct model specification; synthetic data only; recovery degrades under very low observability / high latent dimension; nonlinear case open.
Direction: integrated estimation-and-discovery formulations; nonlinear state estimation (particle/ensemble filters); real datasets (power networks, neural recordings, sensor arrays). Next submission: nonlinear extension, venue TBD, preprint expected spring 2026.