jacobgil / pytorch-grad-cam

Advanced AI Explainability for computer vision. Support for CNNs, Vision Transformers, Classification, Object detection, Segmentation, Image similarity and more.
https://jacobgil.github.io/pytorch-gradcam-book
MIT License
10.16k stars 1.53k forks source link

If I want to visualize a face recognition model, what should I do? I don't seem to see any examples of this #218

Closed SuperCarryFu closed 2 years ago

jacobgil commented 2 years ago

What is the model output? Is it a feature vector, for example used for computing similarity scores between faces? Or something else?

SuperCarryFu commented 2 years ago

Is it a feature vector,Input a face picture, get a feature vector

jacobgil commented 2 years ago

We would have to define a custom target for this, instead of the classifier target. Something like

class FaceRecognitionTarget:
    def __init__(self, target_face_features):
        self.target_face_features = target_face_features
    def __call__(self, model_output):
         return torch.dot(model_output, self.target_face_features)

To see what in the image is similar to a target face, create the target for the target face representation.

Can replace the dot here with L2 norm, cosine similarity, or anything else that is appropriate for computing the similarity between the vectors.

This would be a good example to add as a notebook for this repo. I can add this, but might take me 1-2 weeks to get to it.

SuperCarryFu commented 2 years ago

Thank you very much and I look forward to your updates

---Original--- From: "Jacob @.> Date: Thu, Mar 31, 2022 13:01 PM To: @.>; Cc: @.**@.>; Subject: Re: [jacobgil/pytorch-grad-cam] If I want to visualize a facerecognition model, what should I do? I don't seem to see any examples of this(Issue #218)

We would have to define a custom target for this, instead of the classifier target. Something like class FaceRecognitionTarget: def init(self, target_face_features): self.target_face_features = target_face_features def call(self, model_output): return torch.dot(model_output, self.target_face_features)

To see what in the image is similar to a target face, create the target for the target face representation.

Can replace the dot here with L2 norm, cosine similarity, or anything else that is appropriate for computing the similarity between the vectors.

This would be a good example to add as a notebook for this repo. I can add this, but might take me 1-2 weeks to get to it.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

jacobgil commented 2 years ago

https://github.com/jacobgil/pytorch-grad-cam/blob/master/tutorials/Pixel%20Attribution%20for%20embeddings.ipynb Here you go ;-)