How can I make face recognition faster if I have more than 1M known images?

[khaledabbad]

I tried to use your amazing project for face recognition (for example single unknown image) with my big number of known images (1 Million) but its really slow, this slow because its will load all known images (load_image_file -> then face_encodings) in order to compare with ONE unknown image file.

Any ideas how to speed this process? I was thinking to do face_encodings for all known images then save the 128 as string into apache solr but with no luck as I still need to do compare_faces with all known images:) ... Any suggestions?

[jackweiwang]

Hi @ageitgey
I used the database 'sqlite3'

SELECT * from my_stored_encodings ORDER BY sqrt( power(e1 - TEST_ENCODING_VALUE_0_HERE, 2) + power(e2 - TEST_ENCODING_VALUE_1_HERE, 2) + power(..... etc.... )

I'm sorry to bother you, when I used the database, I found the following error

'Error: no such function: sqrt'

What's the point of using a database if you're not computing Euclidean distances inside a table, or because the database I'm using doesn't support 'sqrt' function?

[ageitgey]

@jackweiwang sqllte is not a traditional database. It's a minimal database that only works for some simple uses. It doesn't have the math functions you'd need to do this calculation in the database, so it wouldn't be a good choice for this kind of use.

[nitishxp]

@khaledabbad can u help me apache solr thing

[arpsyapathy]

Thank you All. I've indexed all images encoding into apache solr and then I managed euclidean distance using solr build-in function dist i.e. http://localhost:8983/solr/mycore/select?q=*:*&fl=dist(2,v_0,v_1,v_3,...,v_127,-0.0621345,0.048437204,0.0839613,...)

So fare, I indexed around 40K images and the query speed very good (17ms without any solr cache)

@khaledabbad Hello! Please let me see your solr config for this and instructions. Thank you!

[deimsdeutsch]

@khaledabbad I am getting around 1.6 seconds for 425,089 images with the default configuration. Any way to get this down.

[arpsyapathy]

@deimsdeutsch Great result! You are using Solr or not? Can you show example of your method?

[anigogo]

@deimsdeutsch Can you tell ur steps how did you use solr.

[shakti-garg]

@ageitgey I have a similar use case of matching one person encoding to millions of ever-growing encodings. Can we use clustering(for ex, kmeans) to create groups of similar image. Then, when new image comes up, we match that first with nearest cluster and then, with nearest encoding within it. The distance method can be the same euclidean or other? Please tell your observations on this approach?

[Dentrax]

@khaledabbad @anigogo @deimsdeutsch @psyapathy

Hey, how should we create a schema.xml design for Apache Solr about this situation? Any examples do you have currently?

Is it okay to create 128 float field like this for my schema.xml file?

<fieldType name="encoding_value" class="solr.FloatPointField" omitTermFreqAndPositions="true" omitNorms="true" required="true" stored="false" docValues="true" multiValued="false"/>

<field name="v_0" type="encoding_value"/>
<field name="v_1" type="encoding_value"/>
...
<field name="v_127" type="encoding_value"/>

[tomriddle54]

Thank you All. I've indexed all images encoding into apache solr and then I managed euclidean distance using solr build-in function dist i.e. http://localhost:8983/solr/mycore/select?q=*:*&fl=dist(2,v_0,v_1,v_3,...,v_127,-0.0621345,0.048437204,0.0839613,...)

So fare, I indexed around 40K images and the query speed very good (17ms without any solr cache)

1. @khaledabbad query means you have taken new image encodings and did euclidean distance using solr Right? and if yes did it took 17 milliseconds to give out prediction.
   2.It's better and helpful for others if you share your apache solr code of save encodings to the database and do the euclidean distance on it.

[hanumantha03]

You could create a database table (postgres, mysql, etc) with 128 columns and store the pre-calculated 1M encodings in that table. Then you could do the compare_faces math using sql against that table to check one face.

There are two approaches for comparing right (1) distance based and (2) knn approach. Q1) Which approach would provide more accuracy? Q2) Can I use cosine similarity for comparing? if yes, what is the accuracy when data size is very large Q3) If KNN-Classifier is giving more accuracy how can we implement it in database? (i am guessing it is tough or near to impossible even then i am asking out of curiosity :) )

[asamedeiros]

Thank you All. I've indexed all images encoding into apache solr and then I managed euclidean distance using solr build-in function dist i.e. http://localhost:8983/solr/mycore/select?q=*:*&fl=dist(2,v_0,v_1,v_3,...,v_127,-0.0621345,0.048437204,0.0839613,...)

So fare, I indexed around 40K images and the query speed very good (17ms without any solr cache)

@khaledabbad can you share the source code if possible?

Thanks

[balajib363]

Hello, This was interesting, I tried storing my features in DB and used it for prediction by calculating Euclidean distance for comparing the images. But here I am facing an issue, I updated in DB only one image feature of a single class image. When I use the same image by little zoomin/zoomout the euclidean distance value is high for the same image. How we can handle this??? Basically here I am not training with multiple images if we are doing how we can store the feature??

Thanks

[thoangnguyen1308]

query = "SELECT first_name, last_name, face_encoding FROM people ORDER BY face_encoding <-> cube(array["+face_encoding_string+"]) LIMIT 1" ERROR: Exception in Tkinter callback Traceback (most recent call last): File "C:\Users\huuthoang\AppData\Local\Continuum\anaconda3\envs\opencv-env\lib\tkinter__init.py", line 1705, in call__ return self.func(*args) File "D:\FaceID\New folder (2)\FaceRecognition.py", line 105, in SubmitImage cur.execute(query) #execute query psycopg2.errors.UndefinedFunction: function cube(text[]) does not exist LINE 1: ...e_encoding FROM people ORDER BY face_encoding <-> cube(array... ^ HINT: No function matches the given name and argument types. You might need to add explicit type casts.

[thoangnguyen1308]

For very large numbers The database searching an encoding may be faster than the application code for comparison (Application code may just loop through entries because we can't hash these encodings for lookup because we need closest match. need confirmation on this?)? I'm not as familiar with the specific details of the API. Additionally, haven't tried but I'm not sure on the management side, how to remove entries from application code so +/- management?

How do I contact you so you can help me resolve the error I'm having

[Pegasus-India]

The formula for euclidean distance is just:

So assuming you had one column for each of the 128 feature values, you could do something like:

SELECT * from my_stored_encodings 
ORDER BY 
      sqrt(
         power(e1 - TEST_ENCODING_VALUE_0_HERE, 2) + 
         power(e2 - TEST_ENCODING_VALUE_1_HERE, 2) + 
         power(..... etc.... 
     )

If you are using Postgresql, you can do more complex things like using it's built-in list data types to store the 128-number encoding in one column and do the comparison using a custom stored function. Just google around for "euclidean distance in sql".

Hi, Thank you for the sample. I have encoded 250 faces in mysql DB and the Face_Recognition is trying to recognize from the db using the above logic.

This is recognizing incorrectly, where as when we use the pics in the code and use "face_distances = face_recognition.face_distance(known_face_encodings, face_encoding)" it works well.

Can you kindly suggest the solution for identifying the correct user name from the db ?

Thanks

[fetfata]

Hi, I tried to use your amazing project for face recognition (for example single unknown image) with my big number of known images (1 Million) but its really slow, I was thinking to do face_encodings database using psycopg2 for all known images. Can anyone share the source code if possible for creating a database and then how to do the query to find of the corresponding face ? thanks

[danielo515]

Sorry for asking on this old thread but, given this formula:

      sqrt(
         power(e1 - TEST_ENCODING_VALUE_0_HERE, 2) + 
         power(e2 - TEST_ENCODING_VALUE_1_HERE, 2) + 
         power(..... etc.... 
     )

is the e1 supposed to be the column name and the TEST_ENCODING_VALUE_0_HERE the literal value?

[danielo515]

Nevermind, it is exactly as I said e1 is the column name, while the left part is the literal value.

Here is a small template (jinja2) that I come with so it is not that tedious to write such query:

{% macro euclidea_distance(column_name) %}
sqrt(
    {% for i, term in encodings -%}
    power({{column_name}}{{ i }} - ({{ term }}), 2) + 
    {% endfor -%}
0)
{% endmacro %}
SELECT  *, {{ euclidea_distance("TERM_") }} as distance 
FROM faces f
HAVING distance < 0.5
ORDER BY distance
LIMIT 30

I found a distance of less than 0.5 gives very good results Then you just provides the terms in an index-value fashion to the template, like this:

template.render(encodings=enumerate(face_encodings))

[PonraJS-21]

I don't think its will be a good idea to save all face features (encodings) in one column, try to add 128 columns with float data type i.e. Column0 float(22,20),Column1 float(22,20), ... Column127 float(22,20).

What field type should I use in solr, in case datatype, where should I mention this

[Andtheworld]

@ageitgey Hey Man, I've fetched the embedding for around 1M images but now I've to load the whole dataset for each query. Can you please suggest a solution in which I can train KNN with already fetched embedding vectors?

Thanks.

[esolsoft]

I am using cube extension and it is good. But if I upload a face that is not in the database it returns the most similar face to the uploaded image with the LIMIT 1 parameter. Is there a way to fix this?

Thanks.

[siddharthverma-1607]

Using a MongoDB you can easily save all the face-encoding in a single document without much hassle of datatypes and columns. I meant to say using a No-SQL for such use case is much better as per my understanding.

[batara666]

@siddharthverma-1607 could you make some example

[MuhammadNumanmn]

Hello can you please share with me your code I will be very thankful to you.

[danielo515]

Hello can you please share with me your code I will be very thankful to you.

Or you can stop asking others to do your work over and over again. Investigate, out effort into it, learn from it and if you have questions (specific questions) ask on stack overflow. I'm sick of lazy people asking for complete solutions without offering anything in return.

[Dancungerald]

I don't think its will be a good idea to save all face features (encodings) in one column, try to add 128 columns with float data type i.e. Column0 float(22,20),Column1 float(22,20), ... Column127 float(22,20).

I don't think its will be a good idea to save all face features (encodings) in one column, try to add 128 columns with float data type i.e. Column0 float(22,20),Column1 float(22,20), ... Column127 float(22,20)

Hey @khaledabbad I'm working on a similar project and I would really appreciate it if you could send me the SQL query for creating the 128 columns you were referring to. And also the query you used to push the face_encodings to the database. I'm using postgressSQL-database.

[xcellenceit]

Can you share source code of this? We will be happy to get assistance on this and sponsor you a beer :)

Thank you All. I've indexed all images encoding into apache solr and then I managed euclidean distance using solr build-in function dist i.e. http://localhost:8983/solr/mycore/select?q=*:*&fl=dist(2,v_0,v_1,v_3,...,v_127,-0.0621345,0.048437204,0.0839613,...)

So fare, I indexed around 40K images and the query speed very good (17ms without any solr cache)

[kekoleg]

The formula for euclidean distance is just:

SELECT * from my_stored_encodings 
ORDER BY 
      sqrt(
         power(e1 - TEST_ENCODING_VALUE_0_HERE, 2) + 
         power(e2 - TEST_ENCODING_VALUE_1_HERE, 2) + 
         power(..... etc.... 
     )

Any advice on what type of index could be used here? It seems to be using seq scan with that ORDER BY and the cost is increasing as the table grows.

[kekoleg]

Any advice on what type of index could be used here?

Just used this example for limiting rows selection and created different indexes: btree (f1, f2 ... f32) and simple index on every f1, f2, f128