What is Anti-Spoofing?

In AI, Facial Recognition (FR) is basically identifying or verifying a person from a digital image or video. FR works by comparing selected facial features from a given image with faces within a database. An application of FR described in this blog is the anti-spoofing of face images. Detection of spoofing from facial recognition is used in applications where people are verified by showing their face in front of a camera.

Face spoofing attacks may take place either by showing a printed photo of a person in front of a camera or by showing the image of a person on a screen such as a mobile phone. Other sophisticated spoofing techniques include video attack or even a 3D mask attack. These attacks are analyzed in terms of descriptors and classifiers.

The following is an overview of these descriptors:

  • Texture descriptors– Faces in mobile in front of the camera produce certain texture patterns that do not exist in real ones. These are texture descriptors. For example:- Local Binary Patterns (LBP), Histograms of oriented gradients (HOG), Deep neural networks (DNN).
  • Motion descriptors– These detect and describe intra-face variations, such as eye blinking, facial expressions, mouth development, head rotation, etc. These help evaluate the consistency of the user interaction within the environment (Liveliness).
  • Frequency descriptors– Artifacts that occur in a spoofing attack (context based) are frequency descriptors.
  • User Interaction (Other)

These descriptors help in the detection of a spoofing medium if present. These can further be used in classifying the given image as a spoofed image or not, as a machine learning problem.


Alternatively, we can use deep learning techniques for anti-spoofing. In this technique, we provide a large number of images as examples of original and spoofed images. The features are identified by learning the patterns from these images, and can thereby spoof images can be detected. This can again be treated as a binary classification problem.


Our methodology

1. Collection of Dataset-

We started with the idea of dividing our dataset into two sets of folders. The dataset was thus categorized into True and False classes; True for the images that are spoofed, and False for the images that are original or not spoofed. For the False images, we used the ‘Northeastern SMILE Lab – Recognizing Faces in the Wild‘ dataset of images on Kaggle, and also some other open-source images. While for the True images, we created a dataset of images inside a mobile frame and then capturing that image. This set was further augmented to form the complete dataset which consisted of about a thousand images each.


2. Split the dataset-

We divided the complete dataset into the following categories:

True class datasets:

Train – (500,)

Validation – (200,)

Test – (200,)

False class datasets:

Train – (500,)

Validation- (200,)

Test – (200,)


3. Pre-processing-

The following is the code implementation for pre-processing.

train_dir = 'training_data'
val_dir = 'validation_data'
test_dir = 'test_data'

train_files = np.concatenate([true_train, false_train])
validate_files = np.concatenate([true_val, false_val])
test_files = np.concatenate([true_test, false_test])

os.mkdir(train_dir) if not os.path.isdir(train_dir) else None
os.mkdir(val_dir) if not os.path.isdir(val_dir) else None
os.mkdir(test_dir) if not os.path.isdir(test_dir) else None

for fn in train_files:
    shutil.copy(fn, train_dir)

for fn in validate_files:
    shutil.copy(fn, val_dir)
    
for fn in test_files:
    shutil.copy(fn, test_dir)
IMG_DIM = (100, 100)

train_files = glob.glob('training_data/*')
train_imgs = [img_to_array(load_img(img, target_size=IMG_DIM)) for img in train_files]
train_imgs = np.array(train_imgs)
train_labels = [fn.split('/')[1].split(' ')[0].strip() for fn in train_files]

validation_files = glob.glob('validation_data/*')
validation_imgs = [img_to_array(load_img(img, target_size=IMG_DIM)) for img in validation_files]
validation_imgs = np.array(validation_imgs)
validation_labels = [fn.split('/')[1].split(' ')[0].strip() for fn in validation_files]

print('Train dataset shape:', train_imgs.shape, 
      '\tValidation dataset shape:', validation_imgs.shape)
Train dataset shape: (1400, 100, 100, 3)  
Validation dataset shape: (512, 100, 100, 3) 
train_imgs_scaled = train_imgs.astype('float32')
validation_imgs_scaled  = validation_imgs.astype('float32')
train_imgs_scaled /= 255
validation_imgs_scaled /= 255

print(train_imgs[0].shape)
array_to_img(train_imgs[0])
 (100, 100, 3) 
batch_size = 10
num_classes = 2
epochs = 100
input_shape = (100, 100, 3)

# encode text category labels
from sklearn.preprocessing import LabelEncoder

le = LabelEncoder()
le.fit(train_labels)
train_labels_enc = le.transform(train_labels)
validation_labels_enc = le.transform(validation_labels)

print(train_labels[695:700], train_labels_enc[695:700])
 ['false', 'true', 'false', 'false', 'true'] [0 1 0 0 1] 

4. Training on VGG16 Architecture-

VGG16 is a convolution neural net (CNN) architecture and is considered to be one of the excellent vision model architecture to date. The model achieves 92.7% top-5 test accuracy in ImageNet, which is a dataset of over 14 million images belonging to 1000 classes. It was submitted to ILSVRC-2014. The following is the architecture of VGG16:

Vgg 16
from keras.applications import vgg16
from keras.models import Model
import keras

vgg = vgg16.VGG16(include_top=False, weights='imagenet', 
                                     input_shape=input_shape)

output = vgg.layers[-1].output
output = keras.layers.Flatten()(output)
vgg_model = Model(vgg.input, output)

vgg_model.trainable = False
for layer in vgg_model.layers:
    layer.trainable = False
    
import pandas as pd
pd.set_option('max_colwidth', -1)
layers = [(layer, layer.name, layer.trainable) for layer in vgg_model.layers]
pd.DataFrame(layers, columns=['Layer Type', 'Layer Name', 'Layer Trainable'])
bottleneck_feature_example = vgg.predict(train_imgs_scaled[0:1])
print(bottleneck_feature_example.shape)
plt.imshow(bottleneck_feature_example[0][:,:,0])
def get_bottleneck_features(model, input_imgs):
    features = model.predict(input_imgs, verbose=0)
    return features
    
train_features_vgg = get_bottleneck_features(vgg_model, train_imgs_scaled)
validation_features_vgg = get_bottleneck_features(vgg_model, validation_imgs_scaled)

print('Train Bottleneck Features:', train_features_vgg.shape, 
      '\tValidation Bottleneck Features:', validation_features_vgg.shape)
Train Bottleneck Features: (1400, 4608)  
Validation Bottleneck Features: (200, 4608) 
from keras.layers import Conv2D, MaxPooling2D, Flatten, Dense, Dropout, InputLayer
from keras.models import Sequential
from keras import optimizers

input_shape = vgg_model.output_shape[1]

model = Sequential()
model.add(InputLayer(input_shape=(input_shape,)))
model.add(Dense(512, activation='relu', input_dim=input_shape))
model.add(Dropout(0.3))
model.add(Dense(512, activation='relu'))
model.add(Dropout(0.3))
model.add(Dense(1, activation='sigmoid'))

model.compile(loss='binary_crossentropy',
              optimizer=optimizers.RMSprop(lr=1e-4),
              metrics=['accuracy'])

model.summary()
history = model.fit(x=train_features_vgg, y=train_labels_enc,
                    validation_data=(validation_features_vgg, validation_labels_enc),
                    batch_size=batch_size,
                    epochs=epochs,
                    verbose=1)

5. Checking the Performance-

f, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 4))
t = f.suptitle('Performance', fontsize=12)
f.subplots_adjust(top=0.85, wspace=0.3)

epoch_list = list(range(1,31))
ax1.plot(epoch_list, history.history['acc'], label='Train Accuracy')
ax1.plot(epoch_list, history.history['val_acc'], label='Validation Accuracy')
ax1.set_xticks(np.arange(0, 31, 5))
ax1.set_ylabel('Accuracy Value')
ax1.set_xlabel('Epoch')
ax1.set_title('Accuracy')
l1 = ax1.legend(loc="best")

ax2.plot(epoch_list, history.history['loss'], label='Train Loss')
ax2.plot(epoch_list, history.history['val_loss'], label='Validation Loss')
ax2.set_xticks(np.arange(0, 31, 5))
ax2.set_ylabel('Loss Value')
ax2.set_xlabel('Epoch')
ax2.set_title('Loss')
l2 = ax2.legend(loc="best")

The performance was checked using the Valuation set. The range of accuracy on the validation set was 0.97 – 1, and that of loss was 0 – 0.10. Then, sav this model as an h5 file.

model.save('first_model.h5')

6. Evaluating the model-

# load dependencies
import glob
import numpy as np
import matplotlib.pyplot as plt
from keras.preprocessing.image import load_img, img_to_array, array_to_img
from keras.models import load_model
import model_evaluation_utils as meu
%matplotlib inline

# load saved models
tl_cnn = load_model('first_model.h5')

# load other configurations
IMG_DIM = (100, 100)
input_shape = (100, 100, 3)
num2class_label_transformer = lambda l: ['false' if x == 0 else 'true' for x in l]
class2num_label_transformer = lambda l: [0 if x == 'false' else 1 for x in l]

# load VGG model for bottleneck features
from keras.applications import vgg16
from keras.models import Model
import keras

vgg = vgg16.VGG16(include_top=False, weights='imagenet', 
                  input_shape=input_shape)
output = vgg.layers[-1].output
output = keras.layers.Flatten()(output)
vgg_model = Model(vgg.input, output)
vgg_model.trainable = False

def get_bottleneck_features(model, input_imgs):
    features = model.predict(input_imgs, verbose=0)
    return features
IMG_DIM = (100, 100)

test_files = glob.glob('test_data/*')
test_imgs = [img_to_array(load_img(img, target_size=IMG_DIM)) for img in test_files]
test_imgs = np.array(test_imgs)
test_labels = [fn.split('/')[1].split(' ')[0].strip() for fn in test_files]

test_imgs_scaled = test_imgs.astype('float32')
test_imgs_scaled /= 255

class2num_label_transformer = lambda l: [0 if x == 'false' else 1 for x in l]
test_labels_enc = class2num_label_transformer(test_labels)
#print(test_labels[0:5], test_labels_enc[0:5])

#le = LabelEncoder()
#le.fit(train_labels)
#test_labels_enc = le.transform(test_labels)


print('Test dataset shape:', test_imgs.shape)
print(test_labels[0:5], test_labels_enc[0:5])
 Test dataset shape: (200, 100, 100, 3) ['true', 'false', 'false', 'true', 'false'] [1, 0, 0, 1, 0] 
import model_evaluation_utils as meu

test_bottleneck_features = get_bottleneck_features(vgg_model, test_imgs_scaled)

from keras.models import load_model
tl_cnn = load_model('first_model.h5')

num2class_label_transformer = lambda l: ['false' if x == 0 else 'true' for x in l]

predictions = tl_cnn.predict_classes(test_bottleneck_features, verbose=0)
predictions = num2class_label_transformer(predictions)
meu.display_model_performance_metrics(true_labels=test_labels, predicted_labels=predictions, 
                                      classes=list(set(test_labels)))

The model performed well against the test set giving an F1 score of 0.995.

Github link- https://github.com/nidhithakkar55/Anti-spoofing/blob/master/Anti_Spoofing.ipynb



For more blogs on Data Science, be sure to check out the following- 🙂

–> Learning Data Science through Fun Demonstrations

–> Industry 4.0, Convergence of IOT and Artificial Intelligence

–> Data Science Problem Formulation through Lego Serious Play (LSP)

–> Design Thinking for Data Science!

–> How to give High-Impact Presentations?

–> Automated Problem Solving in Artificial Intelligence


2 Comments

  1. Nice Blog. The complex approach required for preventing spoofing attack is simplified through this approach. Certainly a good one for L1 requirements.

Leave a Reply

Your email address will not be published. Required fields are marked *

two × four =