CS 280A Project 3 Report

James Peng

1. Defining Correspondences

I defined the corresponding points of images with the provided labeling tool. After labeling, four corner corresponding points were manually added. Then, I generated the triangulation by the Delaunay triangulation algorithm. Below are my images.

source image	target image
source image with triangulation	target image with triangulation

2. Computing the "Mid-way Face"

Computing the mid-way face involves three steps. The first step is to compute the average shape of two images. It could be completed by simply averaging the corresponding points' coordinates. Let the source image be ${\mathbf{I}}_{src}$$\bold I_{src}$ , the target image be ${\mathbf{I}}_{tgt}$$\bold I_{tgt}$ , the corresponding points of the source image be ${\mathbf{P}}_{src}$$\bold P_{src}$ and the corresponding points of the target image be ${\mathbf{P}}_{tgt}$$\bold P_{tgt}$.

After obtaining ${\mathbf{P}}_{avg}$$\bold P_{avg}$ , I applied the Delaunay algorithm on it to get the mid-way face's triangulation.

The second step is to warp both ${\mathbf{I}}_{src}$$\bold I_{src}$ and ${\mathbf{I}}_{tgt}$$\bold I_{tgt}$ into average shape using ${\mathbf{P}}_{avg}$$\bold P_{avg}$ . Nextly, we need to compute the affine transform matrix $\mathbf{M}$$\bold M$ such that $\mathbf{M}{\overrightarrow{p}}_{src}={\overrightarrow{p}}_{mid}$$\bold M \vec p_{src} = \vec p_{mid}$ for each triangle, where $p_{src}$$p_{src}$ and $p_{mid}$$p_{mid}$ are triangles (three vertices, of course) in ${\mathbf{P}}_{src}$$\bold P_{src}$ and ${\mathbf{P}}_{mid}$$\bold P_{mid}$. Rewrite it into matrix form, we have:

Notice that it is equivalent to:

By solving this equation, I obtained $\mathbf{M}$$\bold M$. Therefore, ${\mathbf{M}}^{-1}$$\bold M^{-1}$ can be computed and used for inverse warping (with bi-linear interpolation).

Similarly, we can find ${\mathbf{M}}^{-1}$$\bold M^{-1}$ such that ${\mathbf{M}}^{-1}{\overrightarrow{p}}_{mid}={\overrightarrow{p}}_{tgt}$$\bold M^{-1}\vec p_{mid} = \vec p_{tgt}$ for each triangle as well.

Finally, the third step is averaging the color from two warped image.

The result is shown below:

source image	mid-way face	target image

3. The Morph Sequence

We can generate the morphing sequence with different warp fraction and dissolve fraction in the interval $[0,1]$$[0, \ 1]$. In fact, mid-way face is a special case of morphing sequences with both warp fraction and dissolve fraction equal to $\frac{1}{2}$$\frac12$.

4. The "Mean face" of a population

I used the Dane dataset and computed the mean face for the male. There are 33 males images in the dataset.

Though the facial features of the mean face are clear, the hair was quite blurry. It is possibly because the Dane dataset annotate most of the corresponding points on the face, and there are no corresponding points on the hair.

There is no corresponding point on hair.

This results in when warping images into mean shape, while the facial features are transformed, their hair positions remain, leading to strange outcomes.

Below are some examples that transform the male faces into mean shape. The left column are the original image, and the right column are the images after transformations. The last raw is the photo of myself.

5. Extrapolating from the mean

I did the extrapolation by setting the warp fraction to $1.5$$1.5$ and $-0.5$$-0.5$ .

warp fraction = $1.5$$1.5$, a "very mean shape"	warp fraction = $-0.5$$-0.5$, a "very not mean shape"

Bells and Whistles: PCA analysis

I did the PCA analysis on Dane dataset's male faces. The mean face and the top 15 principle components are shown below:

We can find that for the most important components, they captured the more "general" features while the less important components captured more "specific" details. I pixel-wised did the weighted average between the mean face and the first principle component and got the following image: