Neumann series

Mathematical series

A Neumann series is a mathematical series that sums k-times repeated applications of an operator ${\displaystyle T}$. This has the generator form

${\displaystyle \sum _{k=0}^{\infty }T^{k}}$

where ${\displaystyle T^{k}}$ is the k-times repeated application of ${\displaystyle T}$; ${\displaystyle T^{0}}$ is the identity operator ${\displaystyle I}$ and ${\displaystyle T^{k}:={}T^{k-1}\circ {T}}$ for ${\displaystyle k>0}$. This is a special case of the generalization of a geometric series of real or complex numbers to a geometric series of operators. The generalized initial term of the series is the identity operator ${\displaystyle T^{0}=I}$ and the generalized common ratio of the series is the operator ${\displaystyle T.}$

The series is named after the mathematician Carl Neumann, who used it in 1877 in the context of potential theory. The Neumann series is used in functional analysis. It is closely connected to the resolvent formalism for studying the spectrum of bounded operators and, applied from the left to a function, it forms the Liouville-Neumann series that formally solves Fredholm integral equations.

Properties

Suppose that ${\displaystyle T}$ is a bounded linear operator on the normed vector space ${\displaystyle X}$. If the Neumann series converges in the operator norm, then ${\displaystyle I-T}$ is invertible and its inverse is the series:

${\displaystyle (I-T)^{-1}=\sum _{k=0}^{\infty }T^{k}}$,

where ${\displaystyle I}$ is the identity operator in ${\displaystyle X}$. To see why, consider the partial sums

${\displaystyle S_{n}:=\sum _{k=0}^{n}T^{k}}$.

Then we have

${\displaystyle \lim _{n\rightarrow \infty }(I-T)S_{n}=\lim _{n\rightarrow \infty }\left(\sum _{k=0}^{n}T^{k}-\sum _{k=0}^{n}T^{k+1}\right)=\lim _{n\rightarrow \infty }\left(I-T^{n+1}\right)=I}$

This result on operators is analogous to geometric series in ${\displaystyle \mathbb {R} }$.

One case in which convergence is guaranteed is when ${\displaystyle X}$ is a Banach space and ${\displaystyle |T|<1}$ in the operator norm; another compatible case is that ${\textstyle \sum _{k=0}^{\infty }|T^{k}|}$ converges. However, there are also results which give weaker conditions under which the series converges.

Example

Let ${\displaystyle C\in \mathbb {R} ^{3\times 3}}$ be given by:

${\displaystyle {\begin{pmatrix}0&{\frac {1}{2}}&{\frac {1}{4}}\\{\frac {5}{7}}&0&{\frac {1}{7}}\\{\frac {3}{10}}&{\frac {3}{5}}&0\end{pmatrix}}.}$

We need to show that the matrix norm of C is smaller than unity. Therefore, we calculate

${\displaystyle {\begin{aligned}||C||_{\infty }&=\max _{i}\sum _{j}|c_{ij}|=\max \left\lbrace {\frac {3}{4}},{\frac {6}{7}},{\frac {9}{10}}\right\rbrace ={\frac {9}{10}}<1.\end{aligned}}}$

Thus, we know from the statement above that ${\displaystyle (I-C)^{-1}}$ exists.

Approximate matrix inversion

A truncated Neumann series can be used for approximate matrix inversion. To approximate the inverse of an invertible matrix ${\displaystyle \mathbf {A} }$, we can assign the linear operator as:

${\displaystyle T(\mathbf {x} )=(\mathbf {I} -\mathbf {A} )\mathbf {x} }$

where ${\displaystyle \mathbf {I} }$ is the identity matrix. If the norm condition on ${\displaystyle T}$ is satisfied, then truncating the series at ${\displaystyle n}$, we get:

${\displaystyle \mathbf {A} ^{-1}\approx \sum _{i=0}^{n}(\mathbf {I} -\mathbf {A} )^{i}}$

The set of invertible operators is open

A corollary is that the set of invertible operators between two Banach spaces ${\displaystyle B}$ and ${\displaystyle B'}$ is open in the topology induced by the operator norm. Indeed, let ${\displaystyle S:B\to B'}$ be an invertible operator and let ${\displaystyle T:B\to B'}$ be another operator. If ${\displaystyle |S-T|<|S^{-1}|^{-1}}$, then ${\displaystyle T}$ is also invertible. Since ${\displaystyle |I-S^{-1}T|<1}$, the Neumann series ${\textstyle \sum _{k=0}^{\infty }(I-S^{-1}T)^{k}}$ is convergent. Therefore, we have

${\displaystyle T^{-1}S=(I-(I-S^{-1}T))^{-1}=\sum _{k=0}^{\infty }(I-S^{-1}T)^{k}}$

Taking the norms, we get

${\displaystyle |T^{-1}S|\leq {\frac {1}{1-|I-(S^{-1}T)|}}}$

The norm of ${\displaystyle T^{-1}}$ can be bounded by

${\displaystyle |T^{-1}|\leq {\tfrac {1}{1-q}}|S^{-1}|\quad {\text{where}}\quad q=|S-T|\,|S^{-1}|.}$

Applications

The Neumann series has been used for linear data detection in massive multiuser multiple-input multiple-output (MIMO) wireless systems. Using a truncated Neumann series avoids computation of an explicit matrix inverse, which reduces the complexity of linear data detection from cubic to square.^[1]

Another application is the theory of propagation graphs which takes advantage of Neumann series to derive closed form expressions for transfer functions.

References

1. Wu, M.; Yin, B.; Vosoughi, A.; Studer, C.; Cavallaro, J. R.; Dick, C. (May 2013). "Approximate matrix inversion for high-throughput data detection in the large-scale MIMO uplink". 2013 IEEE International Symposium on Circuits and Systems (ISCAS2013). pp. 2155–2158. doi:10.1109/ISCAS.2013.6572301. hdl:1911/75011. ISBN 978-1-4673-5762-3. S2CID 389966.

• Werner, Dirk (2005). Funktionalanalysis (in German). Springer Verlag. ISBN 3-540-43586-7.