linyq2117 / CLIP-ES

MIT License
175 stars 9 forks source link

loss and gradient backpropagation #23

Closed lk0429 closed 4 months ago

lk0429 commented 4 months ago

Sorry to bother you, I didn't find the loss function and gradient backpropagation process while reading the code. I would like to ask if the initial CAM can be obtained directly? No training required? I see that there is a gradient backpropagation process in the model diagram. If the initial CAM is obtained through training, in which part of the code are the loss function and gradient backpropagation process located? Thanks!

linyq2117 commented 4 months ago

Thanks for your interest!

Our method is based on the pre-trained CLIP model and does not require additional training. We use GradCAM to obtain the CAM, which is based on the gradient of target classes. You can refer to GradCAM for its main idea.

lk0429 commented 4 months ago

Sorry to bother you, I didn't find the loss function and gradient backpropagation process while reading the code. I would like to ask if the initial CAM can be obtained directly? No training required? I see that there is a gradient backpropagation process in the model diagram. If the initial CAM is obtained through training, in which part of the code are the loss function and gradient backpropagation process located? Thanks!

Thanks for your interest!

Our method is based on the pre-trained CLIP model and does not require additional training. We use GradCAM to obtain the CAM, which is based on the gradient of target classes. You can refer to GradCAM for its main idea.