Accuracy for Face Identification, Need some examples for AccuracyCalculator

KevinMusgrave / pytorch-metric-learning

The easiest way to use deep metric learning in your application. Modular, flexible, and extensible. Written in PyTorch.

https://kevinmusgrave.github.io/pytorch-metric-learning/

MIT License

5.95k stars 657 forks source link

Accuracy for Face Identification, Need some examples for AccuracyCalculator #709

Open f-izzat opened 1 month ago

f-izzat commented 1 month ago

I have a dataset consisting 1000 identities, each with a varying number of face images (minimum 3).
I also have a trained face recognition model, say ArcFace to which for every single face image I have its embeddings.
How would I use the AccuracyCalculator, If i were to want to evaluate the model in retrieving the top 5 similar faces to the query?
Would I randomly select a single face image for each identity as a query then calculate its metrics?

Some examples in how the AccuracyCalculator could be used would be much appreciated

Thanks

KevinMusgrave commented 1 month ago

Here's an example of where, for each embedding in query, it finds the nearest 5 neighbors in reference, computes some accuracy metric, and returns the average.

ac = AccuracyCalculator(include=("mean_average_precision",), k=5)
results = ac.get_accuracy(query, query_labels, reference, reference_labels)
print(results["mean_average_precision"])

If your query and reference are the same set, you can just pass in query:

results = ac.get_accuracy(query, query_labels)