Mori–Zwanzig Formulation of Bayesian Filtering

Abstract

Most approximate Bayesian filters in common use project each posterior onto a tractable family, and a projection is judged by the information it discards at the step it is made. We show that this information bounds the cost of a projection only on average: on a single stream, a projection that discards little can cost arbitrarily more in the future. To account for when a projection pays, we apply the Mori–Zwanzig formalism of statistical physics to the filter’s own belief. Under the projection defined by the Fisher metric, every such filter is a Markovian closure of the formalism up to a defect, and its error is the sum of its defects and of the returns of what it discards. For Gaussian references we derive the memory in closed form: for an order-\(k\) closure it decays \(k + 1\) times as fast as the filter forgets its initial condition, or as a power law for a static parameter. From these rates and a square-integrability condition follows a rule for what a filter should resolve: a higher order for smooth non-Gaussianity and an extra component for changepoints. We thereby provide an analysis tool for the error of approximate Bayesian filters.

CitationT.-Y. Tsui, K. Kording, J. Gu, L. Liu. (2026). "Mori–Zwanzig Formulation of Bayesian Filtering."