How to find and anonymize faces using OpenCV and Python

02 November 2020

#opencv #python #face detection #face recognition #anonymization #blurring #anonymize

In order to anonymize faces we need to find them first.

The first method we use is Haar Cascade in OpenCV. It works well and is pretty fast. This example requires you to download this haar cascade xml file from the OpenCV github.

import cv2
import matplotlib.pyplot as plt
import time

start_time = time.time()
face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
img = cv2.imread('usualsuspects.jpg')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
faces = face_cascade.detectMultiScale(gray, 1.1, 5)

for (x, y, w, h) in faces:
    cv2.rectangle(img, (x, y), (x+w, y+h), (0, 0, 255), 3)
print("--- %s seconds ---" % (time.time() - start_time))
plt.figure(figsize=(25,14))
plt.imshow(cv2.cvtColor(img, cv2.COLOR_BGR2RGB))

Completed in 0.3423948287963867 seconds

The second example uses the face_recognition library.

import face_recognition
import time

start_time = time.time()
img = face_recognition.load_image_file('usualsuspects.jpg')
gray = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)
face_locations = face_recognition.face_locations(gray, number_of_times_to_upsample=1, model="cnn")
for face_location in face_locations:
    top, right, bottom, left = face_location
    cv2.rectangle(img, (left, top), (right, bottom), (255, 0, 0), 3)
print("--- %s seconds ---" % (time.time() - start_time))
plt.figure(figsize=(25,14))
plt.imshow(img)
plt.show()

Completed in 11.322885274887085 seconds, that means this library is more than 30x slower. Might not be ideal for real time right? No worries about that for now. Let's go back to our original problem.

Blurring the face is actually pretty easy. We already know the location of the faces, so we can just blur and replace

import cv2
import matplotlib.pyplot as plt
import time
start_time = time.time()
face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
img = cv2.imread('usualsuspects.jpg')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
faces = face_cascade.detectMultiScale(gray, 1.1, 5)

for (x, y, w, h) in faces:
    temp_face = img[y:y+h,x:x+w]
    temp_face = cv2.GaussianBlur(temp_face,(11,11),30)
    img[y:y+h,x:x+w] = temp_face
#     plt.imshow(temp_face)
#     plt.show()
#     cv2.rectangle(img, (x, y), (x+w, y+h), (0, 0, 255), 3)
print("--- %s seconds ---" % (time.time() - start_time))
plt.figure(figsize=(25,14))
plt.imshow(cv2.cvtColor(img, cv2.COLOR_BGR2RGB))

In Poland the media covers a suspect's eyes in photos due to protection laws. It's kinda funny especially when public figures are under arrest. Anyway, let's see if we can automate this process. We need to download this haar cascade file from the OpenCV github for this example. (Note it's different from the previous XML we downloaded)

import cv2
import matplotlib.pyplot as plt
import time
import numpy as np
start_time = time.time()
face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
eye_cascade = cv2.CascadeClassifier("haarcascade_eye.xml")
img = cv2.imread('usualsuspects.jpg')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
faces = face_cascade.detectMultiScale(gray, 1.1, 5)

for (x, y, w, h) in faces:
    temp_face = img[y:y+h,x:x+w]
    roi = gray[y:y+h, x:x+w]
    eyes = eye_cascade.detectMultiScale(roi,1.1,1)
    temp_list = []
    for (ex,ey,ew,eh) in eyes:
        temp_list.append([ex+x,ey+y, ex+x+ew,ey+y+eh])
    boxes = np.asarray(temp_list)
    left = np.min(boxes[:, 0])
    top = np.min(boxes[:, 1])
    right = np.max(boxes[:, 2])
    bottom = np.max(boxes[:, 3])
    cv2.rectangle(img, (left, top), (right, bottom), (0,0,0), -1)
print("--- %s seconds ---" % (time.time() - start_time))
plt.figure(figsize=(25,14))
plt.imshow(cv2.cvtColor(img, cv2.COLOR_BGR2RGB))

Completed in 0.2894630432128906 seconds

So the first thing we notice is that it only detects one eye on Benicio del Toro and Gabriel Byrne. While Gabriel Byrne is complete understandable, we should have a better hit for Benicio. Let's try a alternative approach by using landmarks.

Landmarks is a feature available in the face_recognition package which was used in the second example. Landmarks tell us all the face features like chin, eyes, brows, mouth, etc. This should give us much better results.

import face_recognition
import time
start_time = time.time()
img = face_recognition.load_image_file('usualsuspects.jpg')
gray = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)
face_landmarks_list = face_recognition.face_landmarks(img)
for face in face_landmarks_list:
    pts = np.array(face['left_eye'] + face['right_eye'],np.int32)
    pts = pts.reshape((-1,1,2))
    rect = cv2.minAreaRect(pts)
    box = cv2.boxPoints(rect)
    box = np.int0(box)
    cv2.drawContours(img,[box],0,(0,0,0),-1)
    cv2.drawContours(img,[box],0,(0,0,0),10)
print("--- %s seconds ---" % (time.time() - start_time))
plt.figure(figsize=(25,14))
plt.imshow(img)
plt.show()

Yeh, that's much better! And now you can't totally tell who that is now. Thank got for the protection laws.

Let's draw all of the available landmarks

import face_recognition
import time
start_time = time.time()
img = face_recognition.load_image_file('usualsuspects.jpg')
gray = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)
face_landmarks_list = face_recognition.face_landmarks(img)
for face in face_landmarks_list:
    for f in face:
        pts = np.array(face[f],np.int32)
        pts = pts.reshape((-1,1,2))
        cv2.polylines(img,[pts],False,(0,0,255),1)
print("--- %s seconds ---" % (time.time() - start_time))
plt.figure(figsize=(25,14))
plt.imshow(img)
plt.show()

Completed in 0.4357731342315674 seconds which is super weird because the face_location function took 11 seconds. Very curious, and I'll probably return to investigate a bit later.

Btw, this is how you can create all those cool webcam filters that are popular on chat apps.

How to find and anonymize faces using OpenCV and Python

Further reading