PyChubby - Automated Face Warping

Introduction

We all know the story. You ask a random person on a street to take a photo of you and your friends. After a few moments you happily thank them and go on with your life. After some time you finally sit down for a cup of coffee and check your photos. “My god, why is none of us smiling?!”.

First of all, do not panic. It is your lucky day. If you know how to pip install things there might be hope.

pip install pychubby

Once installed, just write something like this:

import matplotlib.pyplot as plt

from pychubby.actions import Multiple, Smile
from pychubby.detect import LandmarkFace

img_path = 'path/to/img.jpg'
img = plt.imread(img_path)

lf = LandmarkFace.estimate(img)

a = Multiple(Smile(0.15))
lf_new, _ = a.perform(lf)

lf_new.plot(show_numbers=False)

What is PyChubby

Pychubby is an automated face warping tool. Its main goal is to serve as a specialized augmentation interface for deep learning face related tasks. But it might as well be used as a silly face warping tool (see the introduction).

You might wonder why even bother when one can do similar things with Photoshop and other software. The answer is simple - automation. You do not have to locate any landmarks, move them around and then repeat the procedure on each face in every photo.

Popular image augmentation packages like imgaug are general purpose (any kind of image) and do not provide many options when it comes to geometric transformations. pychubby is specialized on human faces and allows for creation of geometric transformations that are:

1. Local (on the face)
2. Smooth (no artifacts)
3. Realistic (to an extent)

In other words one does not augment the whole image but just the faces. The augmentations are realistic and have no artifacts.

Building blocks

The logic of pychubby can be summarized in three blocks

1. Landmark Detection
   • Given a photo, a pretrained landmark detection model predicts 68 landmarks on each face.
2. Reference Space Mapping
   • The landmarks are mapped into a so called reference space. This mapping corrects for possible rotations, translations and scaling in the input image.
3. Manual Action Definition
   • Majority of pychubby actions are defined in the reference space and therefore should yield consistent warpings across different faces. Go to Gallery to see the predefined ones or feel free to define new ones.

Want to know more?

If you are interested in giving pychubby a try or just want to learn more see below a few useful links:

• GitHub
• Documentation

All potential contributors are more than welcome and any feedback is highly appreciated.