Closed sjlljs176 closed 1 month ago
Hi @sjlljs176,
Can you please provide snapshots of the output you are getting? I am running the code on my machine and I am not getting any NaN values in the grads_list. This could be maybe due to the input. So, if you can also please provide sample inputs where you are getting NaN values, it will be easier to reproduce and pinpoint the problem.
Thank you.
the values in activations_and_grads.gradients hava the shape like (1,197,768)
but the reshape_transfrom make it to be a list with nothing
when debug in the get_cam_weights, the "importance" become a tensor with NaN values. I used the images of BUSI dataset as inputs.
The issue was with the gradients shape. I added the fix here.
Thank you for pointing this out @sjlljs176.
Thanks a lot for your time and assistance!
b.t.w, I found the value of loss became < 0 (negative number), when I fine-tuned the BiomedCLIP with MedPix dataset. Is it normal?
It's totally normal. Typical contrastive losses will be bounded between 0 and 1, however when the positive and negative pairs are decoupled the loss will be unbounded and thus can become negative.
thanks a lot.
I noticed that the grads_list was nan when using the BiomedCLIP. Is it normal? I feel weired.
This is the line. https://github.com/HealthX-Lab/MedCLIP-SAM/blob/38e4cadfd607e32d8144427fc9d6f5cc72f64c09/saliency_maps/pytorch_grad_cam/base_cam.py#L192C13-L193C75