HongxinXiang
commented
1 month ago Hi, could you provide the checkpoint you loaded in evaluate.py?
Open AbdulmajeedCE opened 1 month ago
HongxinXiang
commented
1 month ago Hi, could you provide the checkpoint you loaded in evaluate.py?
AbdulmajeedCE
commented
1 month ago SARS-CoV-2.pth.tar https://drive.google.com/file/d/1JnQcxfLsWxaJaAS8fYwCSfFALn2CF17f/view?usp=drive_web hi, I tried to load these two checkpoints 'finetunedALL.pth.tar 'and ' SARS-CoV-2.pth.tar', since it does not work I tried on the pre-trained ' ImageMol.pth.tar' model that is provided in README file even If there are no correct outputs will appear. but I want to see any results.
may this will help you, I think the error says that when the evaluate.py create an empty ResNet15 to load the weights that are in my checkpoints to deal with the loaded weights, it can not because the name of the layers in the checkpoints are different from the standard ResNet15 like ( conv1.weight and so on ), and the name of my checkpoints is like embbeding.conv1.weight something like this. at the end what I mean is it can to mapping the values from the checkpoint to the standard ResNet15. finetunedALL.pth.tar https://drive.google.com/file/d/1mP5NjySGMQr2-SuQdudGVzyvxSCqbd4j/view?usp=drive_web ImageMol.pth.tar https://drive.google.com/file/d/1v7qe-vDHRuUxHdKU9vxdUkvmuAzt6Nh-/view?usp=drive_web
On Mon, Aug 12, 2024 at 5:00 AM HongxinXiang @.***> wrote:
Hi, could you provide the checkpoint you loaded in evaluate.py?
— Reply to this email directly, view it on GitHub https://github.com/HongxinXiang/ImageMol/issues/20#issuecomment-2283000428, or unsubscribe https://github.com/notifications/unsubscribe-auth/BKLDIQJGGUERFZO6JB2C7J3ZRAJLLAVCNFSM6AAAAABMD6GFH2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOBTGAYDANBSHA . You are receiving this because you authored the thread.Message ID: @.***>
HongxinXiang
commented
1 month ago You can try to use the following code to load checkpoint, which supports loading in inconsistent layer names.
if os.path.isfile(args.resume): checkpoint = torch.load(args.resume) ckp_keys = list(checkpoint['model_state_dict']) cur_keys = list(model.state_dict()) model_sd = model.state_dict() if args.image_model == "ResNet18": ckp_keys = ckp_keys[:120] cur_keys = cur_keys[:120] for ckp_key, cur_key in zip(ckp_keys, cur_keys): model_sd[cur_key] = checkpoint['model_state_dict'][ckp_key] model.load_state_dict(model_sd)
AbdulmajeedCE
commented
1 month ago after add this code to evaluate.py I got the same error:
' Architecture: ResNet18 eval_metric: rocauc
/usr/local/lib/python3.10/dist-packages/torchvision/models/_utils.py:208:
UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be
removed in the future, please use 'weights' instead. warnings.warn(
/usr/local/lib/python3.10/dist-packages/torchvision/models/_utils.py:223:
UserWarning: Arguments other than a weight enum or None for 'weights' are
deprecated since 0.13 and may be removed in the future. The current
behavior is equivalent to passing weights=None. warnings.warn(msg) =>
loading checkpoint
'/content/drive/MyDrive/ImageMol/ckpts/SARS-CoV-2.pth.tar' Traceback (most
recent call last): File "/content/drive/MyDrive/OriginalCodes/evaluate.py",
line 93, in main model.load_state_dict(checkpoint) File
"/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line
2189, in load_state_dict raise RuntimeError('Error(s) in loading state_dict
for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for
ResNet: Missing key(s) in state_dict: "conv1.weight", "bn1.weight",
"bn1.bias", "bn1.running_mean", "bn1.running_var", "layer1.0.conv1.weight",
"layer1.0.bn1.weight", "layer1.0.bn1.bias", "layer1.0.bn1.running_mean",
"layer1.0.bn1.running_var", "layer1.0.conv2.weight", "layer1.0.bn2.weight",
"layer1.0.bn2.bias", "layer1.0.bn2.running_mean",
"layer1.0.bn2.running_var", "layer1.1.conv1.weight", "layer1.1.bn1.weight",
"layer1.1.bn1.bias", "layer1.1.bn1.running_mean",
"layer1.1.bn1.running_var", "layer1.1.conv2.weight", "layer1.1.bn2.weight",
"layer1.1.bn2.bias", "layer1.1.bn2.running_mean",
"layer1.1.bn2.running_var", "layer2.0.conv1.weight", "layer2.0.bn1.weight",
"layer2.0.bn1.bias", "layer2.0.bn1.running_mean",
"layer2.0.bn1.running_var", "layer2.0.conv2.weight", "layer2.0.bn2.weight",
"layer2.0.bn2.bias", "layer2.0.bn2.running_mean",
"layer2.0.bn2.running_var", "layer2.0.downsample.0.weight",
"layer2.0.downsample.1.weight", "layer2.0.downsample.1.bias",
"layer2.0.downsample.1.running_mean", "layer2.0.downsample.1.running_var",
"layer2.1.conv1.weight", "layer2.1.bn1.weight", "layer2.1.bn1.bias",
"layer2.1.bn1.running_mean", "layer2.1.bn1.running_var",
"layer2.1.conv2.weight", "layer2.1.bn2.weight", "layer2.1.bn2.bias",
"layer2.1.bn2.running_mean", "layer2.1.bn2.running_var",
"layer3.0.conv1.weight", "layer3.0.bn1.weight", "layer3.0.bn1.bias",
"layer3.0.bn1.running_mean", "layer3.0.bn1.running_var",
"layer3.0.conv2.weight", "layer3.0.bn2.weight", "layer3.0.bn2.bias",
"layer3.0.bn2.running_mean", "layer3.0.bn2.running_var",
"layer3.0.downsample.0.weight", "layer3.0.downsample.1.weight",
"layer3.0.downsample.1.bias", "layer3.0.downsample.1.running_mean",
"layer3.0.downsample.1.running_var", "layer3.1.conv1.weight",
"layer3.1.bn1.weight", "layer3.1.bn1.bias", "layer3.1.bn1.running_mean",
"layer3.1.bn1.running_var", "layer3.1.conv2.weight", "layer3.1.bn2.weight",
"layer3.1.bn2.bias", "layer3.1.bn2.running_mean",
"layer3.1.bn2.running_var", "layer4.0.conv1.weight", "layer4.0.bn1.weight",
"layer4.0.bn1.bias", "layer4.0.bn1.running_mean",
"layer4.0.bn1.running_var", "layer4.0.conv2.weight", "layer4.0.bn2.weight",
"layer4.0.bn2.bias", "layer4.0.bn2.running_mean",
"layer4.0.bn2.running_var", "layer4.0.downsample.0.weight",
"layer4.0.downsample.1.weight", "layer4.0.downsample.1.bias",
"layer4.0.downsample.1.running_mean", "layer4.0.downsample.1.running_var",
"layer4.1.conv1.weight", "layer4.1.bn1.weight", "layer4.1.bn1.bias",
"layer4.1.bn1.running_mean", "layer4.1.bn1.running_var",
"layer4.1.conv2.weight", "layer4.1.bn2.weight", "layer4.1.bn2.bias",
"layer4.1.bn2.running_mean", "layer4.1.bn2.running_var", "fc.weight",
"fc.bias". Unexpected key(s) in state_dict: "arch", "state_dict". During
handling of the above exception, another exception occurred: Traceback
(most recent call last): File
"/content/drive/MyDrive/OriginalCodes/evaluate.py", line 124, in
here is the evaluate.py after adding the code
On Mon, Aug 12, 2024 at 7:50 AM HongxinXiang @.***> wrote:
You can try to use the following code to load checkpoint, which supports loading in inconsistent layer names. if os.path.isfile(args.resume): checkpoint = torch.load(args.resume) ckp_keys = list(checkpoint['model_state_dict']) cur_keys = list(model.state_dict()) model_sd = model.state_dict() if args.image_model == "ResNet18": ckp_keys = ckp_keys[:120] cur_keys = cur_keys[:120] for ckp_key, cur_key in zip(ckp_keys, cur_keys): model_sd[cur_key] = checkpoint['model_state_dict'][ckp_key] model.load_state_dict(model_sd)
— Reply to this email directly, view it on GitHub https://github.com/HongxinXiang/ImageMol/issues/20#issuecomment-2283108271, or unsubscribe https://github.com/notifications/unsubscribe-auth/BKLDIQJNLMAUVNUMFHODXMTZRA5JVAVCNFSM6AAAAABMD6GFH2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOBTGEYDQMRXGE . You are receiving this because you authored the thread.Message ID: @.***>
HongxinXiang
commented
1 month ago Please check whether there is a field with the key model_state_dict in your stored .pth.tar? I cannot access the link you provided because you rejected my application.
AbdulmajeedCE
commented
1 month ago here is a picture provided to you the output is: Checkpoint keys: dict_keys(['arch', 'state_dict'])
[image: Screenshot_49.png]
On Mon, Aug 12, 2024 at 3:30 PM HongxinXiang @.***> wrote:
Please check whether there is a field with the key model_state_dict in your stored .pth.tar? I cannot access the link you provided because you rejected my application.
— Reply to this email directly, view it on GitHub https://github.com/HongxinXiang/ImageMol/issues/20#issuecomment-2283842661, or unsubscribe https://github.com/notifications/unsubscribe-auth/BKLDIQI5WYV6NLFZMEE2WJLZRCTFVAVCNFSM6AAAAABMD6GFH2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOBTHA2DENRWGE . You are receiving this because you authored the thread.Message ID: @.***>
HongxinXiang
commented
1 month ago You can try to use the following code to load checkpoint, which supports loading in inconsistent layer names.
if os.path.isfile(args.resume): checkpoint = torch.load(args.resume) ckp_keys = list(checkpoint['model_state_dict']) cur_keys = list(model.state_dict()) model_sd = model.state_dict() if args.image_model == "ResNet18": ckp_keys = ckp_keys[:120] cur_keys = cur_keys[:120] for ckp_key, cur_key in zip(ckp_keys, cur_keys): model_sd[cur_key] = checkpoint['model_state_dict'][ckp_key] model.load_state_dict(model_sd)
Replace model_state_dict in the code with state_dict
AbdulmajeedCE
commented
1 month ago thank you it works.
but sorry I got an error when trying to use GradCAM visualization, even if I add the in the main class : sys.path.append('/content/drive/MyDrive/OriginalCodes/cam_visualization/').
here are images helping you [image: Screenshot_53.png]
[image: Screenshot_54.png]
On Mon, Aug 12, 2024 at 4:34 PM HongxinXiang @.***> wrote:
You can try to use the following code to load checkpoint, which supports loading in inconsistent layer names. if os.path.isfile(args.resume): checkpoint = torch.load(args.resume) ckp_keys = list(checkpoint['model_state_dict']) cur_keys = list(model.state_dict()) model_sd = model.state_dict() if args.image_model == "ResNet18": ckp_keys = ckp_keys[:120] cur_keys = cur_keys[:120] for ckp_key, cur_key in zip(ckp_keys, cur_keys): model_sd[cur_key] = checkpoint['model_state_dict'][ckp_key] model.load_state_dict(model_sd)
Replace model_state_dict in the code with state_dict
— Reply to this email directly, view it on GitHub https://github.com/HongxinXiang/ImageMol/issues/20#issuecomment-2284011389, or unsubscribe https://github.com/notifications/unsubscribe-auth/BKLDIQN3VNFTBIPOWDGK4CLZRC2WFAVCNFSM6AAAAABMD6GFH2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOBUGAYTCMZYHE . You are receiving this because you authored the thread.Message ID: @.***>
HongxinXiang
commented
1 month ago Sorry, I can't see the images you sent.
AbdulmajeedCE
commented
1 month ago The problem I am facing is in GradCAM Visualization, I opened the main.py in cam_visualization and I append the path “ sys.path.append(*/content/drive/MyDrive/OriginalCodes/cam_visualization/') “ .
When I run the script I got an error message says and I am sure about the paths are correct “ from cam visualization.GradCAM import GradCAM
ModuleNotFoundError: No module named 'cam_visualization' “
On Tue, 13 Aug 2024 at 5:43 AM HongxinXiang @.***> wrote:
Sorry, I can't see the images you sent.
— Reply to this email directly, view it on GitHub https://github.com/HongxinXiang/ImageMol/issues/20#issuecomment-2285238228, or unsubscribe https://github.com/notifications/unsubscribe-auth/BKLDIQIP2JRF5OYS4RHW4S3ZRFXGTAVCNFSM6AAAAABMD6GFH2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOBVGIZTQMRSHA . You are receiving this because you authored the thread.Message ID: @.***>
HongxinXiang
commented
1 month ago try /content/drive/MyDrive/OriginalCodes/ ?
AbdulmajeedCE
commented
1 month ago thank you so much every thing is working, but I want mention that I faced the same problem in evaluate.py in loading the model then I used the same code you provided it to me.
and I want to make sure that everything is good, does warning affect the output? this /usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py:1373: UserWarning: Using a non-full backward hook when the forward contains multiple autograd Nodes is deprecated and will be removed in future versions. This hook will be missing some grad_input. Please use register_full_backward_hook to get the documented behavior. warnings.warn("Using a non-full backward hook when the forward contains multiple autograd Nodes " execute completed.
On Tue, Aug 13, 2024 at 7:06 AM HongxinXiang @.***> wrote:
try /content/drive/MyDrive/OriginalCodes/ ?
— Reply to this email directly, view it on GitHub https://github.com/HongxinXiang/ImageMol/issues/20#issuecomment-2285300259, or unsubscribe https://github.com/notifications/unsubscribe-auth/BKLDIQM36YKZQHHTYGLCRXDZRGA4BAVCNFSM6AAAAABMD6GFH2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOBVGMYDAMRVHE . You are receiving this because you authored the thread.Message ID: @.***>
HongxinXiang
commented
1 month ago AbdulmajeedCE
commented
4 weeks ago hi, I hope you are doing well.
I want to ask you about the t-SNE ( red and blue dots ), and for clustering which I mean all the figures appeared in the paper.
thank you.
On Tue, Aug 13, 2024 at 8:31 AM HongxinXiang @.***> wrote:
- You can use the methods mentioned above to solve the checkpoint loading problem.
- The warning may be due to version reasons and I guess it should have no effect.
— Reply to this email directly, view it on GitHub https://github.com/HongxinXiang/ImageMol/issues/20#issuecomment-2285370122, or unsubscribe https://github.com/notifications/unsubscribe-auth/BKLDIQP7XJKI5VNOQSRJO53ZRGK3XAVCNFSM6AAAAABMD6GFH2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOBVGM3TAMJSGI . You are receiving this because you authored the thread.Message ID: @.***>
HongxinXiang
commented
4 weeks ago If you want to generate a t-SNE plot, use the following demo code: ` import numpy as np import matplotlib.pyplot as plt from scipy.stats import gaussian_kde
x = np.random.normal(size=100) y = x * 3 + np.random.normal(size=100)
xy = np.vstack([x,y]) z = gaussian_kde(xy)(xy)
fig, ax = plt.subplots() ax.scatter(x, y, c=z, s=100, s=4, cmap='Reds') # or cmap='Blues' plt.show() `
AbdulmajeedCE
commented
3 weeks ago thank you so much,
could you please provide me the dataset of the finetuned model that is mentioned in the README file for BBBP ?
the reason I asked you to have the same dataset you used because to see the exact results and may improving it.
On Thu, Aug 15, 2024 at 7:28 AM HongxinXiang @.***> wrote:
If you want to generate a t-SNE plot, use the following demo code: ` import numpy as np import matplotlib.pyplot as plt from scipy.stats import gaussian_kde
x = np.random.normal(size=100) y = x * 3 + np.random.normal(size=100)
xy = np.vstack([x,y]) z = gaussian_kde(xy)(xy)
fig, ax = plt.subplots() ax.scatter(x, y, c=z, s=100, s=4, cmap='Reds') # or cmap='Blues' plt.show() `
— Reply to this email directly, view it on GitHub https://github.com/HongxinXiang/ImageMol/issues/20#issuecomment-2290559554, or unsubscribe https://github.com/notifications/unsubscribe-auth/BKLDIQOM7JYTKCWC5Q4ODFTZRQU6ZAVCNFSM6AAAAABMD6GFH2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOJQGU2TSNJVGQ . You are receiving this because you authored the thread.Message ID: @.***>
HongxinXiang
commented
2 days ago Sorry for the late reply, the data can be found at this link: https://drive.google.com/file/d/1IdW6J6tX4j5JU0bFcQcuOBVwGNdX7pZp/view?usp=sharing
hello there, I hope you are doing well. please help me on this issue that I am facing,
1- I downloaded the pretrained model 'You can download pre-trained model and push it into the folder ckpts/'. which is in 'Finetuning' section in README file.
2- then I used finetune.py to finetune the pretrained model that I just download on SARS-CoV-2 dataset which is provided to you. until here everything is good and I got the finetuned model on this dataset ( but note that may this information help you later, after start finetuning ) this warning appears:
' Architecture: ResNet18 eval_metric: rocauc /usr/local/lib/python3.10/dist-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead. warnings.warn( /usr/local/lib/python3.10/dist-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or
Nonefor 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passingweights=None. warnings.warn(msg) => loading checkpoint '/content/drive/MyDrive/KACSTProject/ckpts/SARS-CoV-2.pth.tar''3-but still i got the finetune model, now if i want to use the evaluate.py on the same dataset and the same finetuned model i got this error:
Architecture: ResNet18 eval_metric: rocauc /usr/local/lib/python3.10/dist-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead. warnings.warn( /usr/local/lib/python3.10/dist-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or
Nonefor 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passingweights=None. warnings.warn(msg) => loading checkpoint '/content/drive/MyDrive/KACSTProject/ckpts/SARS-CoV-2.pth.tar' Traceback (most recent call last): File "/content/drive/MyDrive/KACSTProject/evaluate.py", line 93, in main model.load_state_dict(checkpoint) File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 2189, in load_state_dict raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for ResNet: Missing key(s) in state_dict: "conv1.weight", "bn1.weight", "bn1.bias", "bn1.running_mean", "bn1.running_var", "layer1.0.conv1.weight", "layer1.0.bn1.weight", "layer1.0.bn1.bias", "layer1.0.bn1.running_mean", "layer1.0.bn1.running_var", "layer1.0.conv2.weight", "layer1.0.bn2.weight", "layer1.0.bn2.bias", "layer1.0.bn2.running_mean", "layer1.0.bn2.running_var", "layer1.1.conv1.weight", "layer1.1.bn1.weight", "layer1.1.bn1.bias", "layer1.1.bn1.running_mean", "layer1.1.bn1.running_var", "layer1.1.conv2.weight", "layer1.1.bn2.weight", "layer1.1.bn2.bias", "layer1.1.bn2.running_mean", "layer1.1.bn2.running_var", "layer2.0.conv1.weight", "layer2.0.bn1.weight", "layer2.0.bn1.bias", "layer2.0.bn1.running_mean", "layer2.0.bn1.running_var", "layer2.0.conv2.weight", "layer2.0.bn2.weight", "layer2.0.bn2.bias", "layer2.0.bn2.running_mean", "layer2.0.bn2.running_var", "layer2.0.downsample.0.weight", "layer2.0.downsample.1.weight", "layer2.0.downsample.1.bias", "layer2.0.downsample.1.running_mean", "layer2.0.downsample.1.running_var", "layer2.1.conv1.weight", "layer2.1.bn1.weight", "layer2.1.bn1.bias", "layer2.1.bn1.running_mean", "layer2.1.bn1.running_var", "layer2.1.conv2.weight", "layer2.1.bn2.weight", "layer2.1.bn2.bias", "layer2.1.bn2.running_mean", "layer2.1.bn2.running_var", "layer3.0.conv1.weight", "layer3.0.bn1.weight", "layer3.0.bn1.bias", "layer3.0.bn1.running_mean", "layer3.0.bn1.running_var", "layer3.0.conv2.weight", "layer3.0.bn2.weight", "layer3.0.bn2.bias", "layer3.0.bn2.running_mean", "layer3.0.bn2.running_var", "layer3.0.downsample.0.weight", "layer3.0.downsample.1.weight", "layer3.0.downsample.1.bias", "layer3.0.downsample.1.running_mean", "layer3.0.downsample.1.running_var", "layer3.1.conv1.weight", "layer3.1.bn1.weight", "layer3.1.bn1.bias", "layer3.1.bn1.running_mean", "layer3.1.bn1.running_var", "layer3.1.conv2.weight", "layer3.1.bn2.weight", "layer3.1.bn2.bias", "layer3.1.bn2.running_mean", "layer3.1.bn2.running_var", "layer4.0.conv1.weight", "layer4.0.bn1.weight", "layer4.0.bn1.bias", "layer4.0.bn1.running_mean", "layer4.0.bn1.running_var", "layer4.0.conv2.weight", "layer4.0.bn2.weight", "layer4.0.bn2.bias", "layer4.0.bn2.running_mean", "layer4.0.bn2.running_var", "layer4.0.downsample.0.weight", "layer4.0.downsample.1.weight", "layer4.0.downsample.1.bias", "layer4.0.downsample.1.running_mean", "layer4.0.downsample.1.running_var", "layer4.1.conv1.weight", "layer4.1.bn1.weight", "layer4.1.bn1.bias", "layer4.1.bn1.running_mean", "layer4.1.bn1.running_var", "layer4.1.conv2.weight", "layer4.1.bn2.weight", "layer4.1.bn2.bias", "layer4.1.bn2.running_mean", "layer4.1.bn2.running_var", "fc.weight", "fc.bias". Unexpected key(s) in state_dict: "arch", "state_dict".During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/content/drive/MyDrive/KACSTProject/evaluate.py", line 124, in
main(args)
File "/content/drive/MyDrive/KACSTProject/evaluate.py", line 95, in main
checkpoint = torch.load(args.resume)["model_state_dict"]
KeyError: 'model_state_dict' '.
4- if you need any other information I am happy to provide it to you, thanks a lot. SARS-CoV-2.zip