Face Recognition Security System based on Internet of Things

Project Background

We all have a smartphone with a facial unlock feature. But many of us still use basic old mechanical locks to secure our houses, security locks, etc. Wouldn't it be amazing to install a facial recognition system feature in our houses like we see in the movies? Here, in this project, I will show a prototype of such a face recognition security system with all information about the process and the coding parts and hardware devices that you can also follow to build such a system.

Hardware components

• Bolt IoT Bolt WiFi Module × 1    
  

• LDR, 5 megohm × 1    

• USB-A to Mini-USB data cable × 1    

• Buzzer × 1    

• LED (general purpose) × 1    

• Breadboard (general purpose) × 1    

  
  

Software applications and online services

• Bolt IoT Bolt Cloud
  

• OpenCV    

• Microsoft Visual Studio 2017

  
  

Hand tools and manufacturing machines

• Servo Motor, Premium Male/Male Jumper
  

• Premium female/male extension jumper, 40 x 6" (150 mm)

  
  

Project Requirements

The first device the user will interact with is our LDR - Light Detection Resistor.

Face recognition will only kick in when there is an obstacle to light intensity, otherwise, if we connect the face recognition system directly, it will start consuming memory and power.

Now, if the LDR observes any obstruction in the light intensity, it will send the information to Bolt Cloud and our Python code written in the system will observe that information. python will start the webcam to scan for authorized persons or intruders. If there is an authorized person in the camera's view, it will detect and close the circuit on port 1, to which our relay is connected, and which connects the motor to the power supply. The motor here will act as a pulley and pull the metal rod out of the hole. And it will also send a telegraphic message to the authorities granting access with an authorized name and time. If there is an intruder in the camera's view, the circuit consisting of a buzzer will shut down.

That's all we have to do here. Now, let's look at the step-by-step process of building this project.

 Procedure

Hardware setup

Connect one of the LDR terminals to the 3v3 port of the Bolt IoT module, and the other to the "A0" analog port. (This is because the A0 terminal of the Bolt IoT module is the only port in the Bolt IoT that can be used to get similar input values).

Connect the positive terminal of the buzzer to the "0" digital port, and connect the other terminal to the GND port. (This is because digital port "0" is used for both inputs and outputs, in this case we use it as an output port to power the buzzer through port "0").

Connect the positive terminal of the LED to the "1" digital port and the negative port to the GND port. (This is because digital port "1" is used for both input and output, in this case we use it as an output port to power the LED through port "1"). Here, I don't have a relay, so just to demonstrate I'm using LEDs.

Use a breadboard to connect these circuits as shown in the diagram. (Using a breadboard is not mandatory, we are using it so that we can easily make changes to the circuit later if needed and also for simplicity).

Now, connect the USB cable to the bolt module and the other end to the power supply via your laptop or charger.

This concludes the hardware setup. For more information on the LDR and Bolt IoT modules, click on the embedded links.

Note: All of these connections are made in the Bolt IoT device and can be purchased from the link below.

Software Setup

First, we need to install python on our system, otherwise "*.py" will not open. You can download python from here . Then make sure your pip is up to date by typing the following command at the command prompt.

Now, you need to install some libraries to run the project. Otherwise, you will receive import errors. The libraries needed for this project are

Open base library: pip install opencv-python

NumPy library: pip install numpy

Face recognition: pip install face-recognition

Now, you need to install some libraries to run this project. Otherwise, you will receive import errors. The libraries needed for this project are the open CV libraries: NumPy library: face_recognition: pip install opencv-python

pip install numpy

pip install face-recognition

Procedure

Create a python - conf.py: Enter the device API and device ID in this file.

Second, create a python file - embedding.py: This file is used to create the code for the authorized person and create a pickle file for this data to be used for face recognition.

Third, create a python file - recognition.py: This file is the main file that forms the face detection program for the authorization function. This is the main file that contains the IoT implementation to add functionality through Bolt IoT devices.

How it works.

• First, the authorized person must create his/her face code by opening the embedding.py file. Then enter their name and any number and click "s" 5 times to get 5 samples of their face. This is the pickle file where all the codes will be saved in the same directory where the file embedding.py is located.

• Now when the person wants to open the door. He/she has to put his/her hand on the LDR, which will signal to Bolt Cloud via the API that the LDR value has changed.

• Now the python code will receive these signals and launch the webcam to start the facial recognition process.

• The facial encoding will then be created and compared to the encoding of the previously created pickle file.

• Now, if the person is an authorized person, the signal will be passed to the IoT device indicating that the person is an authorized person and closing the circuit on port 1, which in our case has an LED connected, but a relay with a stepper motor will be attached in the live application. Then, a message will also be sent to the authorities indicating who has been granted access and for how long.

• If the person is not an authorized person, a signal that the person is not authorized will be passed to the IoT device. To alert the intruder, the circuitry on port 0 will close, passing current to the buzzer, which will start ringing. And a telegraphic message about the intruder will also be sent to the authorities along with a time stamp.

  
  

Original research on the face recognition technology used in this project.

This project is based on original research published in Modern Face Recognition with Deep Learning. The original research here shows that the captured face images are passed through the deep learning algorithm Hog's Algorithm, which follows the following steps.

• First, the captured image is converted into a black and white image.

• Then each pixel is analyzed according to the darkness of each corresponding pixel.

• An image gradient is created in which all pixels are converted to arrows and then the strongest arrows form an image gradient called Hog's Image.

• This image of a pig's face is compared with various training to detect faces in face recognition.

• Nevertheless, the algorithm will not be able to identify whether the face is tilted or not.

• Therefore, the facial sign estimation algorithm is now used to tilt the image to center the lips, eyes and nose. To do this, we will use the basic method of tilting like a rotation and scaling parallel lines, called affine transformation.

• The basic idea behind this is that we have 68 specific points on our face, located on the lips, the outer structure of the eyes, the top of the chin, etc.

• Then an image is created consisting of these 68 points.

• This image is then analyzed to detect the face.

• But it would be very time consuming to compare one image with several other images each time.

• So, now what we want to do is to create one measurement from the training image and then save those measurements. And when it's time to analyze the new image instead of comparing the image with other training images. We just compare the measurements of the new image with the previously saved measurements.

• Now for measurements, we use a deep convolutional neural network to generate 128 measurements for each face.

• But still it is how the computer recognizes these images. For this purpose, we compare 3 images.

• The first one is a previously saved image of a known person.

• The second one is a new image of a known person.

• The third is an image of an unknown person.

• And if the new image measurements are more similar to the measurements of the previously saved image of the known person. Then the computer will know that the new image is of the known person.

• In ML, these 128 measurements of the face image are called embedding.

• But in order to get the accuracy, we need to find the best 128 measurements that we can use to get perfect face recognition.

• To get these perfect 128 measurements, we need to train these images for millions of people, millions of times, which is very time consuming.

• So, the people at Open Face published their training study. used in the project to achieve accuracy.

• This research was used as a reference to create the face recognition library that we used in this project.

  
  

This is the way to build a prototype of an LDR-activated IoT-based face recognition security system.

This is just a prototype, and that's why we're using LEDs here. You can use relays and motors to add unlocking capabilities to your project. In addition, you can add telegraphic messaging capabilities to the project to warn authorities of intruders by attaching their facial images.