Pretrianed models perform poorly

[MutazAbueisheh]

Describe the bug

The issue we are facing is that when we use the pretrained models we get a very poor performance as per the attached image where EfficientNet B4 was used (accuracy is less than 1%)

Please be informed that we have the same case for all other models for Sig53 dataset. We also have the same case for WBsig53 models.

Are we using the pretrained model to obtain the weights in the right way ? or is there an issue with the model?

Thank you!

To Reproduce Steps to reproduce the behavior: We followed these steps in order to use the pretrained models:

• Generate a small portion of the dataset, with the option qa True in the terminal

• Running 02_example_sig53_classifier_using_pretrained Notebook with following settings:

  1-When choosing and calling the model inside the notebook, pretrained value is set to be True 2-The cell of training the model was not executed (and the one that follows which is for selecting the best checkpoint

Expected behavior We expect to have a good performance close to the one showed in the related papers to Torchsig

Screenshots

Operating System: Ubuntu

Versions of the env: Versions

Versions of relevant libraries: [pip3] numpy==1.26.4 [pip3] pytorch-lightning==2.2.1 [pip3] torch==2.0.1 [pip3] torchmetrics==1.3.2 [pip3] torchsig==0.4.1 [pip3] torchsummary==1.5.1 [pip3] torchvision==0.15.2 [pip3] triton==2.0.0 [conda] Could not collect

[MattCarrickPL]

@MutazAbueisheh thanks for reporting, we are tracking this internally now.