Information Geometry

Statistical manifolds

Let \((\mathcal X, \mathcal F)\) be a measure space with a \(\sigma\)-finite measure \(\mu\). \(L^1(\mu)\) denotes the Banach space of \(\mu\)-integrable functions, and

\[\mathcal P(\mathcal X) = \left\{ p \in L^1(\mu) \;\middle|\; p>0, \int_{\mathcal{X}} p\, d\mu=1 \right\}\]

denotes the family of all densities of probability measures equivalent to \(\mu\) (note: \(\mathcal P(\mathcal X)\) is naturally a topological space with the induced \(L^1\) topology). The expectation of a function \(f\) with respect to some \(p\) is \(E_p [f] \coloneqq \int_{\mathcal X} f(x) p(x) d\mu(x)\), if it exists.

A (regular) statistical model parametrized by some open set \(\Theta \subset \mathbb R^d\) is a subspace \(M \subset \mathcal P(\mathcal X)\) together with a homeomorphism \(\Theta \to M\) denoted by \(\theta \mapsto p_\theta\), that satisfies the following conditions:

  1. for every \(x \in \mathcal X\), the map \(\theta \mapsto p_\theta(x)\) is \(C^{\infty}\),
  2. let \(\ell_\theta(x) \coloneqq \log p_\theta(x)\) and \(\partial_i \coloneqq \frac{\partial}{\partial \theta_i}\). For every \(\theta \in \Theta\), the set of functions \(\left\{ \partial_i \ell_\theta : i \in \{1,\dots,d\} \right\}\) is linearly independent,
  3. all partial derivatives of \(p_\theta\) and \(\ell_\theta\) commute with integrals, and
  4. all moments \(E_\theta[\partial_{i_1} \ell_\theta \dots \partial_{i_n} \ell_\theta ]\) exist.

If, furthermore, \(M\) has the structure of a \(C^\infty\) manifold completely parametrized by \(\theta\), then \(M\) is said to be a statistical manifold.