180 Proj3

Overview

In this project, we create a 'morph' (a warp of image shapes and a cross-dissolve) animation of one face transitioning into a different face. We compute the mean (extrapolated) face of a population of faces and use it to synthesize new faces. Additionally, we generate a video showing the morphing process from one image to another. The dataset used in this project is the FEI Face dataset, which contains 400 images of faces. Link to the dataset

Part 1. Defining Correspondences

Workflow

We first define pairs of corresponding points on the two images by hand. Here is the Correspondence Tool.
Then, resize the image to the same size, and use the provided tool above to get the corresponding points of the two images, to calculate the average points of the two images.
Finally, apply the fun function compute_delaunay to get the Delaunay triangulation of the average points.

Outputs

Keypoints and triangulation

Part 2: Computing the "Mid-way Face"

Workflow

We first obtain the triangulation of the average points between the two images from part 1.
Next, I matched the average points with the corresponding points in img1 and img2 based on the triangulations.
We calculate the affine transformation matrix from one image to another using the function computeAffine(tri1_pts, tri2_pts).
Finally, we use polygons to mask all the pixels and apply each inverse matrix to the original image to get the final result.
Averaging the colors together(cross-dissolved)

Here is the original A and B images as well as the image of the mid-way face that you got.

Original Image A

Original Image B

Midway Face Image

Part 3: The Morph Sequence

In this section, I implemented the function `morphed_im = morph(im1, im2, im1_pts, im2_pts, tri, warp_frac, dissolve_frac)` which returns a morphed image for given warp and dissolve fractions.
First, images `im1` and `im2` are warped to the average shape determined by `warp_frac`. Then, they are cross-dissolved based on the `dissolve_frac` to produce the final morphed image.
A sequence of images is generated using a for loop that iterates through different values of `warp_frac` and `dissolve_frac`, appends each morphed image to a list, and saves the list as a GIF using the PIL library.

Outputs

Morph Sequence

Part 4. The "Mean face" of a population

In this part, I am using face images in FEI Face dataset, and there are provided points for each image stored as pts file.
I first read the images and points, and then calculate the average points of all the images to get the triangulation in order to warp each img to the avg img
Then dissolve the images use for loop get the final result. I normalized the image to make sure the result is good.

Outputs

img_arr[2]

warped_img_arr[2]

img_arr[6]

warped_img_arr[6]

img_arr[8]

warped_img_arr[8]

img_arr[5]

warped_img_arr[5]

avg_face

avg_face_smile

After get the average face, I aligned George's face to the mean face, to get face warped into the average geometry, the average face warped into the img geometry.

George to average

average to George

Part 5: Caricatures - Extrapolating from the mean

This part is similar to the previous section. The average points are calculated using the formula: `points = mypoints + a * (avg_face_points - mypoints)`, where `a` can be less than 0 or greater than 1.