The issue we are facing is that when we use the pretrained models we get a very poor performance as per the attached image where EfficientNet B4 was used (accuracy is less than 1%)
Please be informed that we have the same case for all other models for Sig53 dataset.
We also have the same case for WBsig53 models.
Are we using the pretrained model to obtain the weights in the right way ? or is there an issue with the model?
Thank you!
To Reproduce
Steps to reproduce the behavior:
We followed these steps in order to use the pretrained models:
Generate a small portion of the dataset, with the option qa True in the terminal
Running 02_example_sig53_classifier_using_pretrained Notebook with following settings:
1-When choosing and calling the model inside the notebook, pretrained value is set to be True
2-The cell of training the model was not executed (and the one that follows which is for selecting the best checkpoint
Expected behavior
We expect to have a good performance close to the one showed in the related papers to Torchsig
Screenshots
Operating System: Ubuntu
Versions of the env:
Versions
Versions of relevant libraries:
[pip3] numpy==1.26.4
[pip3] pytorch-lightning==2.2.1
[pip3] torch==2.0.1
[pip3] torchmetrics==1.3.2
[pip3] torchsig==0.4.1
[pip3] torchsummary==1.5.1
[pip3] torchvision==0.15.2
[pip3] triton==2.0.0
[conda] Could not collect
Describe the bug
The issue we are facing is that when we use the pretrained models we get a very poor performance as per the attached image where EfficientNet B4 was used (accuracy is less than 1%)
Please be informed that we have the same case for all other models for Sig53 dataset. We also have the same case for WBsig53 models.
Are we using the pretrained model to obtain the weights in the right way ? or is there an issue with the model?
Thank you!
To Reproduce Steps to reproduce the behavior: We followed these steps in order to use the pretrained models:
Running 02_example_sig53_classifier_using_pretrained Notebook with following settings:
1-When choosing and calling the model inside the notebook, pretrained value is set to be True 2-The cell of training the model was not executed (and the one that follows which is for selecting the best checkpoint
Expected behavior We expect to have a good performance close to the one showed in the related papers to Torchsig
Screenshots
Operating System: Ubuntu
Versions of the env: Versions
Versions of relevant libraries: [pip3] numpy==1.26.4 [pip3] pytorch-lightning==2.2.1 [pip3] torch==2.0.1 [pip3] torchmetrics==1.3.2 [pip3] torchsig==0.4.1 [pip3] torchsummary==1.5.1 [pip3] torchvision==0.15.2 [pip3] triton==2.0.0 [conda] Could not collect