Inner Product

An inner product is a generalization of the dot product. Both a dot product and an inner product take in two vectors and output a scalar. The dot product is found by multiplying the $k$-th component of the first vector by the $k$-th component of the second, and summing all these products.

$$\vec{u}\cdot \vec{v}=u_1{v}_1+u_2{v}_2+...u_n{v}_n=\sum _{k=1}^n{u}_k{v}_k$$

An inner product can be any other function so long as it satisfies the requirements below. For example, we could define a vector of $n$ scalar weights, $w$, and apply these weights to the two vectors:

$$⟨\vec{u},\vec{v}⟩=\sum _{k=1}^n{w}_k{u}_k{v}_k$$

Note that if we define the inner product to include these weights, then all inner products and operations relying on the inner product used thereafter will include these weights implicitly.

Definition

Within a given vector space $V$, an inner product is any vector function, denoted $⟨,⟩$ or $⟨\vec{u},\vec{v}⟩$, which satisfies the following three axioms for some arbitrary vectors $\vec{u}$, $\vec{v}$, and $\vec{w}$ in $V$. The three axioms are slightly different for vector spaces in $\mathbb{R}$ vs. in $\mathbb{C}$.

Real Inner Products

Positive-definiteness

$$⟨\vec{u},\vec{u}⟩\ge 0⟨\vec{u},\vec{u}⟩=0\text{only if}\vec{u}=0$$

Bilinearity

Recall that linearity is composed of additivity, $f(x+y)=f(x)+f(y)$, and homogeneity, $f(ax)=a\cdot f(x)$. So for an inner product, we must have,

$$⟨a\vec{u},\vec{v}⟩=a\cdot ⟨\vec{u},\vec{v}⟩⟨\vec{u},b\vec{v}⟩=b\cdot ⟨\vec{u},\vec{v}⟩$$ $$⟨a\vec{u}+b\vec{v},\vec{w}⟩=⟨a\vec{u},\vec{w}⟩+⟨b\vec{v},\vec{w}⟩⟨\vec{u},a\vec{v}+b\vec{w}⟩=⟨\vec{u},a\vec{v}⟩+⟨\vec{v},b\vec{w}⟩$$

for any scalars $a$ and $b$.

Symmetry

$$⟨\vec{u},\vec{v}⟩=⟨\vec{v},\vec{u}⟩$$

Complex Inner Products

Positive-definiteness

$$⟨\vec{u},\vec{u}⟩\ge 0⟨\vec{u},\vec{u}⟩=0\text{only if}\vec{u}=0$$

”Sesquilinearity”

A word which means “one-and-a-half linearity”. Bilinearity cannot hold at the same time as symmetry and positive-definiteness for complex inputs. For example, if we have $⟨i\vec{u},i\vec{u}⟩$, then positive-definiteness insists that this product be greater than or equal to 0, but if we apply bilinearity, we have $⟨i\vec{u},i\vec{u}⟩=i\cdot i\cdot ⟨\vec{u},\vec{u}⟩=(-1)\cdot ⟨\vec{u},\vec{u}⟩<0$. Therefore we need to modify the requirement of linearity so that for one (not both) of the inputs, if linearity is applied, we get the conjugate of the scalar factor out instead of just the scalar factor itself.

$$⟨a\vec{u},\vec{v}⟩=a\cdot ⟨\vec{u},\vec{v}⟩⟨\vec{u},b\vec{v}⟩=\bar{b}\cdot ⟨\vec{u},\vec{v}⟩=b^{\ast }\cdot ⟨\vec{u},\vec{v}⟩$$

or vice versa with $b$ remaining linear and $a\to \bar{a}$. Then, positive-definiteness holds: $⟨i\vec{u},i\vec{u}⟩=i\cdot \bar{i}\cdot ⟨\vec{u},\vec{u}⟩=(i)(-i)\cdot ⟨\vec{u},\vec{u}⟩=(1)\cdot ⟨\vec{u},\vec{u}⟩<0$.

Additivity remains the same as for real numbers:

$$⟨a\vec{u}+b\vec{v},\vec{w}⟩=⟨a\vec{u},\vec{w}⟩+⟨b\vec{v},\vec{w}⟩⟨\vec{u},a\vec{v}+b\vec{w}⟩=⟨\vec{u},a\vec{v}⟩+⟨\vec{v},b\vec{w}⟩$$

Conjugate Symmetry

$$⟨\vec{u},\vec{v}⟩=\overline{⟨\vec{v},\vec{u}⟩}=⟨\vec{v},\vec{u}{⟩}^{\ast }$$

Uses

An inner product is used to define the following properties. These properties are normally fundamental to vector calculus, and most vector spaces are also inner product spaces where these properties are well defined. In vector space which are not also inner product space, these properties become largely meaningless or cannot be defined.

Vector Length or Norm

$$||\vec{u}||:=\sqrt{⟨\vec{u},\vec{u}⟩}$$

This property is called the norm of the inner product, and is also the equation used to define the length of a vector. A vector space that can define a norm using its inner product is a normed vector space. If we’ve established length as a property, a number of others flow from this, including angle, distance, orthogonality, projection, etc.

Distance Between Vectors

$$d(\vec{u},\vec{v}):=||\vec{u}-\vec{v}||=\sqrt{⟨\vec{u}-\vec{v},\vec{u}-\vec{v}⟩}$$

This is also called “metric”, and a space which has a metric is a metric space. In a metric space we can talk about points being “close” or “far” from each other.

Angle Between Vectors

$$cos(\theta ):=\frac{⟨\vec{u},\vec{v}⟩}{||\vec{u}||||\vec{v}||}\text{or}\theta :={\cos }^{-1}(\frac{⟨\vec{u},\vec{v}⟩}{||\vec{u}||||\vec{v}||})$$

Vector Orthogonality

$$\vec{u}\perp \vec{v}\text{only when}⟨\vec{u},\vec{v}⟩=0$$

Pythagorean Theorem

If $\vec{u}\perp \vec{v}$, then,

$$||\vec{u}|{|}^2+||\vec{v}|{|}^2=||\vec{u}+\vec{v}|{|}^2$$

Inner Product Space

A vector space that has an inner product associated with it is an inner product space. On some vector space $V$ we can define multiple inner products, and so give rise to many inner product spaces from the same underlying vector space. There can also be a vector space with no inner product.

Examples of Inner Product Spaces

The real numbers $\mathbb{R}$, where the inner product is $⟨x,y⟩=xy$.

Euclidean space, ${\mathbb{R}}^n$, where the inner product is the dot product, $⟨(x_1,x_2,...x_n),(y_1,y_2,...y_n)⟩=x_1{y}_1+x_2{y}_2+...x_n{y}_n$.

References

1. Real and Complex Inner Products - Columbia University
2. Inner Product - Wolfram Mathworld
3. Inner Product Space - Wikipedia