[DMP 2024]: Implementation of Facial Recognition based attendance system for teachers in the Government schools

[convegenius-tech]

Ticket Contents

Description

Government schools in India often face challenges in accurately recording and monitoring teacher attendance, leading to inefficiencies in resource allocation and management. Traditional methods of attendance tracking, such as manual sign-in sheets, are prone to errors, time-consuming, and lack real-time monitoring capabilities. Therefore, there is a need for an automated, reliable, and efficient offline solution to streamline the process of teacher attendance management in government schools.

The objective of this project is to evaluate and implement open-source facial recognition models to develop a demo application showcasing the reliability of the chosen model for teacher attendance tracking in Indian government schools. Students will research available open-source facial recognition models, assess their suitability for the intended use case, and select the most appropriate one. The selected model will then be integrated into a prototype demo application, providing a tangible demonstration of its accuracy and reliability in real-world scenarios.

Goals & Mid-Point Milestone

Goals

• [ ] Research on open-source facial recognition model works online/offline both.
• [ ] Prepare or outsource benchmark dataset and evaluate models on defined criteria.
• [ ] Explore and create user registration pipeline to store facial data and landmarks.
• [ ] [Goals Achieved By Mid-point Milestone]
• [ ] Implement efficient face lookup algorithm with confidence scores.
• [ ] Create a demo offline/online application to facilitate user registration and lookup with geo tagging.

Setup/Installation

No response

Expected Outcome

• Functional prototype of the demo application featuring the facial recognition model, in offline/online for teacher facial registration/onboarding, attendance capturing with admin support to track and view associated analysis.
• Benchmark report comparing the performance of available open-source facial recognition models against predefined criteria.
• Documentation outlining the integration process, system architecture, and user guidelines for future reference and maintenance.

Acceptance Criteria

• Highly accurate attendance (above a 95% threshold) capturing solution.
• The application functions in both online and offline environments (if applicable).
• Intuitive user interface for easy registration, attendance capture and admin support.
• Robust solution with implemented privacy measures to protect teachers' facial data and ensure compliance with relevant data protection regulations.

Implementation Details

• Use open source libraries.
• Our platform backend is in node.js but ai solution can be deployed in python with an endpoint to use.

Mockups/Wireframes

No response

Product Name

Swift Attendance

Organisation Name

ConveGenius AI

Domain

⁠Education

Tech Skills Needed

AWS, Computer Vision, Docker, Flask, Machine Learning, Node.js, Python

Mentor(s)

@kavita-gsphk @addyag93

Category

Machine Learning, Mobile, Research

[MadhukeshSingh]

Hi there, @convegenius-tech, The problem statement provided is very clear and lies in my areas of interest as I'm passionate about machine learning and keen on joining this project.

Here's a bit about myself: I am Madhukesh Singh, currently studying at the National Institute of Technology, Hamirpur, in my third year.

My experience includes working on image processing, computer vision, and object detection in satellite imagery during my internship as an AI developer at DRDO DYSL.AI.

Is there a preferred method for communicating with the mentors? I'm eager to contact you and explore how I can contribute.

[rroy1920]

hi @convegenius-tech , @kavita-gsphk , @addyag93, could you please assign me this project because i am interested as well as experienced in mobile application development.

Here's a bit about myself: I am Rahul Roy, currently pursuing btech in my 3rd year at NIT Hamirpur, Himachal Pradesh.

I have developed different mobile application before including a recent Ram mandir app .

[Nithinlakavath3424]

Hi there, @convegenius-tech , The problem statement provided is very clear and lies in my areas of interest as I'm passionate about machine learning and keen on joining this project.

Here's a bit about myself: I am LAKAVATH NITHIN, currently studying at the Vignana Bharathi Institute of Technology, Hyderabad, in my third year, in the specialization of artificial intelligence and machine learning.

My experience includes working on image processing, computer vision, object detection, conversion of image into grayscale images. And one of the best project I did is face recognition with appropriately 90% accurate solution.

[Dilpreet0501]

Hi @convegenius-tech , this problem statement is clear to me and its tech stacks alligns with my skills.

Here's a bit about myself: I am Dilpreet Kaur Bhatia currently pursuing B.TECH in 3rd year at NIT Raipur.

My experience includes VNN, image processing, object detection and computer vision. I have also build some models based on these.

I would like to communicate with mentors and would like to know how can I contribute .

[Yrastogi]

Hello! @convegenius-tech My name is Yash Rastogi, and I'm currently pursuing my B.Tech in Computer Science at Meerut Institute of Engineering and Technology. I am deeply interested in contributing to the "Swift Attendance" project by ConveGenius AI. The opportunity to work on an innovative solution using open-source facial recognition technology for teacher attendance management in government schools aligns perfectly with my background and passion for leveraging technology for social impact.

To select the most suitable open-source facial recognition model for the "Swift Attendance" project, focus on models like OpenCV (utilizing dlib or OpenCV's built-in algorithms), face_recognition, DeepFace, and OpenFace. Each model based on accuracy, resource efficiency, and offline capability to ensure it meets the project's needs for accurate, efficient, and deployable teacher attendance tracking in government schools. Please send your Update on My mail- seemayash012@gmail.com or via phone number - +91 7983164675

[MadhukeshSingh]

PR #2 please check for my facial recognition app @convegenius-tech

[sowmiyaM-1]

Hi @convegenius-tech, @kavita-gsphk , @addyag93.

About myself: I am sowmiya currently studying EEE pre-final year at kongunadu College of Engineering and Technology, trichy. I have worked on opening, YOLO, tensorflow, image processing and object detection integrating with the embedded and IoT. I would like to contribute to this project as I have the hope that I can do it.

[Sayanjones]

Hey @convegenius-tech, I'm Sayan and interested in contributing to the project. My skills in computer vision, machine learning, and Python align perfectly with the project's requirements.

I've been following the project description and believe my experience with Docker and performance optimization can be valuable in ensuring the solution scales efficiently, especially for offline functionality. I'm a strong proponent of open-source development and eager to contribute by developing demos that showcase the chosen facial recognition model's effectiveness.

[Jugrajsinghbali]

Hello @convegenius-tech,@kavita-gsphk , @addyag93

I'm Jugraj Singh, currently studying at Noida Institute of Engineering. I have experience in image processing, computer vision, and object detection from my AI developer internship at Codsoft. The project aligns with my interests in machine learning, and I'm eager to contribute and look forward to your response

Thank you

[memyselfandglitch]

Hi @convegenius-tech , I am Deveshi Singh, a third year undergraduate student at IIT(ISM) Dhanbad. I have worked on a product development facility in my college where I worked extensively with computer vision and Raspberry Pi to develop a blind aid for visually impaired individuals. I am very passionate about machine learning and the field of computer vision. I have worked extensively with the MERN stack and I am familiar with NoSQL databases. I am really looking forward to making a contribution to this project. I would love to work on this project and learn new things in the process. Regards Deveshi

[aman-raj-srivastva]

Hi there, @convegenius-tech , The problem statement provided is very clear and lies in my areas of interest as I'm passionate about machine learning and keen on joining this project.

Here's a bit about myself: I am AMAN RAJ, currently studying at the VIVEKANANDA INSTITUTE OF PROFESSIONAL STUDIES, DELHI, in my BTECH second year, in the specialization of artificial intelligence and machine learning.

My experience includes working on image processing, computer vision, object detection, conversion of image into grayscale images.

[Pratikdate]

Hey Hi @convegenius-tech , It's my loving problem statement that I had already created. you should check link : https://github.com/Pratikdate/face_recognition_fierbase

But in top of that, I implemented Firebase real time database update.

I think here also for online we should use Firebase ML kit. As my tech stack, i am very interested in working on this project.

Can i start working on it immediately ?

[amamisha59]

Hello,

I am Amisha, a third-year B.Tech student with a keen interest in frontend development and a growing passion for artificial intelligence and machine learning, particularly in the field of facial recognition. I am excited about the opportunity to contribute to the development of an automated attendance tracking system for government schools in India. Looking forward to contributing to this project!

[Mansi08git]

Hello Mentor @convegenius-tech @kavita-gsphk @addyag93 I am Mansi Soni , a second year undergraduate , currently pursuing B.Tech in CSE . I have a keen interest in machine learning and artificial intelligence and currently I am working on computer vision . I am interested to collaborate in the above project . I found myself best fit to contribute to this project as I am already working on face-recognition project which is 70% completed . Looking forward eagerly to contribute to this project and explore the world of AI.

[himanshuktgkb]

Hii @convegenius-tech,@kavita-gsphk , @addyag93

My Self Himanshu Tiwari, I currently studying at NIT Jalandhar in CSE branch. I have experience in computer vision, and object detection ,YOLO , Flask , Python. I have also build some projects on these topics . You can check out in my GitHub Profile . This project aligns with my interests in machine learning ,computer vision , Flask and Python , and I'm eager to contribute and look forward to your response.

[krishnarathore12]

Hello @convegenius-tech I am Krishna Rathore undergraduate student at IIT Patna. I have a deep passion for AI and also recent advancements in NLP make me wonder about the future of AI. Here are some of my achievements.

• worked under Prof. Asif Ekbal in top 2% researcher by Stanford
• Smart India Hackathon 2023 finalist
• Bronze medal holder in Inter-IIT Tech Competition 2023

I have used face-recognition library to identify the faces in the images and drawn rectangle around them. Model is pretty lightweight and gives reliable results.

[Srilekha-09]

Hello @convegenius-tech, I am Madupu Srilekha from BVRIT.I am very much interested to contribute to this project. I have a prior experience of building the projects using the computer vision, Machine learning ,cloud computing and in python programming.I have participated in many hackathons which deals with this subjects like airtel iq hackathon.In the recent times I have worked on the same topics related to attendance systems using Intel software which is working really well..Looking forward eagerly to contribute to this project and explore the world of ML.

[divyanshu1990]

Hello @convegenius-tech I am Divyanshu pursing Btech in computer science engineering (second year). As the project is very related to my summer project because I done this type of project in my summer vacation, so I can much contribute to it.

Regards Divyanshu

[Shorya-Dixit]

Proposal-1.pdf

[prakharsingh-74]

@convegenius-tech @addyag93 @kavita-gsphk may I know why you don't do the interview for this precious project?

[kavita-gsphk]

@prakharsingh-74 The interview process has been concluded. If you did not receive an interview invitation, it means you were not selected for further steps. Thank you for your participation.

[prakharsingh-74]

@kavita-gsphk can you tell me the area of improvement?

[kavita-gsphk]

Weekly Goals

Week 1

• [x] Finalise the dataset
• [x] Steup and understand the working of deepface

Week 2

• [x] Prepare the dataset
• [x] Create the pipeline for testing with the finalised dataset
• [x] Finalise and test the top 10 configuration

Week 3

• [x] Test the top 5 configurations with and without the specs dataset
• [x] Test how models perform with 1. Different image quality 2. Rotated images 3. Blurriness 4. Different resolution

Week 4

• [x] Test the top 4 configuration with different resolution datasets, and finalise the resolution
• [x] Clean the dataset collected with mvp app and test it with top 4 configuration.

Week 5

• [x] Test different augmentation method/finalize pre processing steps

Week 6

• [x] Test the finalized configuration and preprocessing steps on whole dataset
• [x] Find out scenario where model is failing

Week 7

• [x] Research and test on model who can detect failing scenarios

Week 8

• [x] Finalise the model who can detect failing scenarios
• [x] Prepare the user guideline based on research have been done till now
• [x] Research on Offline solution

Week 9

• [x] Test the accuracy drop for downsampling
• [x] Continue research for Offline solution

Week 10

• [x] Continue research and model testing for Offline solution

Week 11

• [x] Continue research and model testing for Offline solution

Week 12

• [x] Test on required images for registration and how can we combine multiple embedding
• [x] Continue research and model testing for Offline solution

[gupta-tilak]

Weekly Learnings & Updates

Week 1

• Finalised Indian-Facial-Database-Highlighting-Spectacle-Problem dataset for the project.
• Understood the working of deepface and studied the preprocessing pipeline which includes normalisation, resizing and alignment.

  Week 2

• Studied various research papers on Facial Recognition and performance of various existing open-source models for the task.
• Preprocessed and cleaned the dataset as per the project requirements keeping 30 images of each individual.
• Prepared 5000 testing combinations with 1500 test images.
• Created the testing pipeline for deepface with 12 initial model-detector-distance-alignment configurations.

  Week 3

• Finalised the top 5 configurations in terms of accuracy and time taken to process the results.
• Built the data augmentation pipeline to test the selected configurations on params like varying image quality, providing a rotation angle to the images, introducing some blurriness in the images and trying out different resolutions.

  Week 4

• After performing various tests selected the recognition models and the detector backends for the task.
• To further test these models, diversified the testing data with 2 new datasets and the data from the mvp app.
• Performed tests to choose an optimum image resolution for the facial recognition task.
• Proceeded with cleaning of the mvp app data in stages and recorded configuration performance metrics at each stage, signifying the instructions our app should show to the end user while registering and capturing attendance.
• Started working over various preprocessing and augmenting steps that can be integrated in the pipeline for improving the performance.

  Week 5

• Researched over various techniques prevalent to improve facial recognition accuracy like Pseudorandom Pix
• el Placement, Denoising, grayscale conversion, resolution upscaling using GFP-GAN and histogram equalisation.
• On an average increase in accuracy observed was 1.3%.
• Performed test with the new pipeline with Gaussian integrated on all the image combinations without specs and achieved the highest ever accuracy.
• Finalised the preprocessing steps and the model configuration.

  Week 6

• Fine-tuned the threshold value for the selected model - detector pair with 2 approaches.
• Approach 1 revolved around statistical techniques for selecting the threshold covers maximum data and approach 2 revolved around the C4.5 algorithm similar to the tree based on adaboost algorithm.
• Tested the model with the fine-tuned threshold on both the datasets.
• Also, fine tuned the parameters involved in the preprocessing steps.
• Proceeded with the False Positive / Negative analysis to draw common patterns where the model is failing.

  Week 7

• Analysing the false positive and negative results from the 5000 image combinations gave the general observation that Sure, here's the refined version of the points:
• Analysed the false positive and negative results from 5000 image combinations, identifying three main failure conditions:
  1. Images too dark, rendering faces unrecognisable.
  2. Eyes closed or not centered on the screen.
  3. Tilted faces leading to irregular facial structures.
• Researched and experimented with various models and methods to address these issues.
• Proposed using average grayscale intensity as a measure for illumination in images.
• Investigated pupil detection models for potential application in our use case.

  Week 8

• After investigating various available options for the problems of eyes closed, eyes not centered on the screen and extreme dark images which have unrecognisable faces proposed the frame freeze approach for first 2 problems and average illumination intensity check before registering images for further processes.
• Finalised the freeze frame that would be used to guide the user where to locate the face before capturing the image.
• Finalised the illumination intensity threshold balancing the tradeoff between skin complexion and background effects.
• Prepared the set of instructions/guidelines that need to be displayed to the user.

  Week 9

• Figured out a potential issue with the resize operation in the current testing pipeline.
• Implemented a new resize function which maintains the aspect ratio of the original image and fills the difference between resized image and target size by the black pixels.
• Researched for a high-res dataset and tested the new pipeline.
• Finalised the aspect ratio and the image resolution to be used in the frontend of the attendance app.

  Week 10

• Researched over face recognition and detection models that can be used on edge device with a limited system capabilities.
• Finalised 2 detector and 2 recogntion state of the art models in there respective domains for the task.
• Performed an in-depth analysis of these models with other open source models in the space of constrained speed and complexity.

  Week 11

• Focused on integrating EdgeFace and FaceBoxes, two cutting-edge models in the facial recognition domain, optimised for edge computing environments.
• Successfully built an offline facial recognition pipeline by bringing in the two models EdgeFace and FaceBoxes.
• Completed the repo and it can be referenced at https://github.com/gupta-tilak/faceboxes-edgeface-FR
• Initiated testing to assess the accuracy and computational efficiency of the new model configuration for the offline solution and generated initial testing results.

  Week 12

• Conducted research to compare the effectiveness of single-image versus multiple-image template embeddings in the vector database, aiming to develop more robust template embeddings.
• Determined the optimal number of images to store in the database by evaluating the cost-performance tradeoff, supported by comprehensive research findings.
• Tested various arithmetic methods for combining embeddings tailored to our datasets, optimizing for accuracy and efficiency.
• Finalized both the optimal number of images and the best arithmetic methods for combining embeddings based on extensive testing and analysis.
• Evaluated multiple model configurations for the offline solution, focusing on enhancing performance and improving overall results.