search

TorchDSP / torchsig

TorchSig is an open-source signal processing machine learning toolkit based on the PyTorch data handling pipeline.
MIT License
174 stars 40 forks source link

Can't load pretrained models anymore #239

Open nsbruce opened 4 months ago

nsbruce commented 4 months ago

Describe the bug Since the torch version was updated, the provided pretrained weights no longer match the efficientnet model and can't be loaded in.

To Reproduce

from torchsig.models.iq_models.efficientnet.efficientnet import efficientnet_b4

convnet = efficentnet_b4(pretrained=True, path='/path/to/efficientnet_b4_online.pt')

Output

Traceback (most recent call last):                                                                                             
  File "<frozen runpy>", line 198, in _run_module_as_main                                                                      
  File "<frozen runpy>", line 88, in _run_code                                                                                 
  File "/train.py", line 135, in <module>                                            
    my_model = train_mymodel()
               ^^^^^^^^^^^^^^                                                                                              
  File "/.venv/lib/python3.11/site-packages/click/core.py", line 1157, in __call__          
    return self.main(*args, **kwargs)                                                                                          
           ^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                          
  File "/.venv/lib/python3.11/site-packages/click/core.py", line 1078, in main              
    rv = self.invoke(ctx)                                                                                                      
         ^^^^^^^^^^^^^^^^                                                                                                      
  File "/.venv/lib/python3.11/site-packages/click/core.py", line 1434, in invoke            
    return ctx.invoke(self.callback, **ctx.params)                                                                             
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                             
  File "/.venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke             
    return __callback(*args, **kwargs)                                                                                         
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                         
  File "train.py", line 127, in train_mymodel                                       
    model = MyModel()
    ^^^^^^^^^^^^^^^^
  File "/mymodel.py", line 28, in __init__                                            
    self.encoder = Encoder()                                                                                         
    ^^^^^^^^^^^^^^^^^^^^                                                                                       
  File "/encoder.py", line 8, in __init__                                            
    self.convnet = efficientnet_b4(pretrained=pretrained, path=path)                                                           
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                           
  File "/.venv/lib/python3.11/site-packages/torchsig/models/iq_models/efficientnet/efficient
net.py", line 285, in efficientnet_b4                                                                                          
    mdl.load_state_dict(torch.load(path))                                                                                      
  File "/.venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 2189, in load_
state_dict                                                                                                                     
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(                                                  
RuntimeError: Error(s) in loading state_dict for EfficientNet:

Missing key(s) in state_dict: "bn1.weight", "bn1.bias", "bn1.running_mean", "bn1.running_var", "blocks.0.0.bn1.weight",
 "blocks.0.0.bn1.bias", "blocks.0.0.bn1.running_mean", "blocks.0.0.bn1.running_var", "blocks.0.0.bn2.weight", "blocks.0.0.bn2.b
ias", "blocks.0.0.bn2.running_mean", "blocks.0.0.bn2.running_var", "blocks.0.1.bn1.weight", ..., "bn2.running_mean", "bn2.running_var".

Unexpected key(s) in state_dict: "bn1.bn.weight", "bn1.bn.bias", "bn1.bn.running_mean", "bn1.bn.running_var", "bn1.bn.n
um_batches_tracked", "blocks.0.0.bn1.bn.weight", "blocks.0.0.bn1.bn.bias", "blocks.0.0.bn1.bn.running_mean", "blocks.0.0.bn1.bn
.running_var", "blocks.0.0.bn1.bn.num_batches_tracked", "blocks.0.0.bn2.bn.weight", "blocks.0.0.bn2.bn.bias", "blocks.0.0.bn2.b
n.running_mean", "blocks.0.0.bn2.bn.running_var", "blocks.0.0.bn2.bn.num_batches_tracked", "blocks.0.1.bn1.bn.weight",..., "bn2.bn.running_var", "bn2.bn.num_batches_tracked".

Note I shortened the output by a lot since there are many mismatched keys between these dictionaries.

Expected behavior model loads in correctly

Some possible solutions:

nsbruce commented 4 months ago

It looks like the version bump came from @pvallance - do you have any input on what the best approach here is?

nsbruce commented 4 months ago

https://github.com/huggingface/pytorch-image-models/issues/2150 https://github.com/huggingface/pytorch-image-models/issues/2215 https://huggingface.co/timm/swinv2_tiny_window16_256.ms_in1k/discussions/1

These references got me one step closer (past the above error). I adjusted the timm.create_model() calls in efficientnet.py. I'm now getting the following (related) error:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/train.py", line 135, in <module>
    my_model = train_mymodel()
                   ^^^^^^^^^^^^^^
  File "/.venv/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/train.py", line 127, in train_sigclr
    model = MyModel()
  File "/mymodel.py", line 28, in __init__
    self.encoder = Encoder()
                   ^^^^^^^^^^^^^^^^^^^
  File "/encoder.py", line 8, in __init__
    self.convnet = efficientnet_b4(pretrained=pretrained, path=path)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.11/site-packages/torchsig/models/iq_models/efficientnet/efficientnet.py", line 276, in efficientnet_b4
    timm.create_model(
  File "/.venv/lib/python3.11/site-packages/timm/models/_factory.py", line 117, in create_model
    model = create_fn(
            ^^^^^^^^^^
  File "/.venv/lib/python3.11/site-packages/timm/models/efficientnet.py", line 1913, in efficientnet_b4
    model = _gen_efficientnet(
            ^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.11/site-packages/timm/models/efficientnet.py", line 651, in _gen_efficientnet
    model = _create_effnet(variant, pretrained, **model_kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.11/site-packages/timm/models/efficientnet.py", line 368, in _create_effnet
    model = build_model_with_cfg(
            ^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.11/site-packages/timm/models/_builder.py", line 418, in build_model_with_cfg
    load_pretrained(
  File "/.venv/lib/python3.11/site-packages/timm/models/_builder.py", line 215, in load_pretrained
    state_dict[weight_name] = adapt_input_conv(in_chans, state_dict[weight_name])
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.11/site-packages/timm/models/_manipulate.py", line 259, in adapt_input_conv
    O, I, J, K = conv_weight.shape
    ^^^^^^^^^^
ValueError: not enough values to unpack (expected 4, got 3)
pvallance commented 4 months ago

The default efficientnets from timm are 2D. We have utilities for converting these models to 1D (typically used for the narrowband datasets). It looks like you might be trying to load the weights of a 1D network into a 2D model. Could you pass along your script snippets that are using torchsig and/or the datasets?

Phil Vallance

RF Research Team

(240) 373-6289

From: Phillip Vallance Sent: Saturday, July 13, 2024 12:25:54 PM To: TorchDSP/torchsig; TorchDSP/torchsig Cc: Mention Subject: Re: [TorchDSP/torchsig] Can't load pretrained models anymore (Issue #239)

Nicholas,

Are you using the 1-D efficientnet from torchsig?

Phil Vallance

RF Research Team

(240) 373-6289

From: Nicholas Bruce @.***> Sent: Friday, July 12, 2024 7:55:09 PM To: TorchDSP/torchsig Cc: Phillip Vallance; Mention Subject: Re: [TorchDSP/torchsig] Can't load pretrained models anymore (Issue #239)

huggingface/pytorch-image-models#2150https://github.com/huggingface/pytorch-image-models/issues/2150 huggingface/pytorch-image-models#2215https://github.com/huggingface/pytorch-image-models/issues/2215 https://huggingface.co/timm/swinv2_tiny_window16_256.ms_in1k/discussions/1

These references got me one step closer (past the above error). I adjusted the timm.create_model() calls in efficientnet.py. I'm now getting the following (related) error:

Traceback (most recent call last): File "", line 198, in _run_module_as_main File "", line 88, in _run_code File "/train.py", line 135, in my_model = train_mymodel() ^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/click/core.py", line 1157, in call return self.main(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) ^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, ctx.params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke return __callback(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/train.py", line 127, in train_sigclr model = MyModel() File "/mymodel.py", line 28, in init self.encoder = Encoder() ^^^^^^^^^^^^^^^^^^^ File "/encoder.py", line 8, in init self.convnet = efficientnet_b4(pretrained=pretrained, path=path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/torchsig/models/iq_models/efficientnet/efficientnet.py", line 276, in efficientnet_b4 timm.create_model( File "/.venv/lib/python3.11/site-packages/timm/models/_factory.py", line 117, in create_model model = create_fn( ^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/timm/models/efficientnet.py", line 1913, in efficientnet_b4 model = _gen_efficientnet( ^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/timm/models/efficientnet.py", line 651, in _gen_efficientnet model = _create_effnet(variant, pretrained, model_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/timm/models/efficientnet.py", line 368, in _create_effnet model = build_model_with_cfg( ^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/timm/models/_builder.py", line 418, in build_model_with_cfg load_pretrained( File "/.venv/lib/python3.11/site-packages/timm/models/_builder.py", line 215, in load_pretrained state_dict[weight_name] = adapt_input_conv(in_chans, state_dict[weight_name]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/timm/models/_manipulate.py", line 259, in adapt_input_conv O, I, J, K = conv_weight.shape ^^^^^^^^^^ ValueError: not enough values to unpack (expected 4, got 3)

— Reply to this email directly, view it on GitHubhttps://github.com/TorchDSP/torchsig/issues/239#issuecomment-2226541580, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AUN3HRQZU4GO52C7QYQNCTDZMBUF3AVCNFSM6AAAAABKZVB5ZCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMRWGU2DCNJYGA. You are receiving this because you were mentioned.Message ID: @.***>

pvallance commented 4 months ago

Nicholas,

Are you using the 1-D efficientnet from torchsig?

Phil Vallance

RF Research Team

(240) 373-6289

From: Nicholas Bruce @.***> Sent: Friday, July 12, 2024 7:55:09 PM To: TorchDSP/torchsig Cc: Phillip Vallance; Mention Subject: Re: [TorchDSP/torchsig] Can't load pretrained models anymore (Issue #239)

huggingface/pytorch-image-models#2150https://github.com/huggingface/pytorch-image-models/issues/2150 huggingface/pytorch-image-models#2215https://github.com/huggingface/pytorch-image-models/issues/2215 https://huggingface.co/timm/swinv2_tiny_window16_256.ms_in1k/discussions/1

These references got me one step closer (past the above error). I adjusted the timm.create_model() calls in efficientnet.py. I'm now getting the following (related) error:

Traceback (most recent call last): File "", line 198, in _run_module_as_main File "", line 88, in _run_code File "/train.py", line 135, in my_model = train_mymodel() ^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/click/core.py", line 1157, in call return self.main(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) ^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, ctx.params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke return __callback(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/train.py", line 127, in train_sigclr model = MyModel() File "/mymodel.py", line 28, in init self.encoder = Encoder() ^^^^^^^^^^^^^^^^^^^ File "/encoder.py", line 8, in init self.convnet = efficientnet_b4(pretrained=pretrained, path=path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/torchsig/models/iq_models/efficientnet/efficientnet.py", line 276, in efficientnet_b4 timm.create_model( File "/.venv/lib/python3.11/site-packages/timm/models/_factory.py", line 117, in create_model model = create_fn( ^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/timm/models/efficientnet.py", line 1913, in efficientnet_b4 model = _gen_efficientnet( ^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/timm/models/efficientnet.py", line 651, in _gen_efficientnet model = _create_effnet(variant, pretrained, model_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/timm/models/efficientnet.py", line 368, in _create_effnet model = build_model_with_cfg( ^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/timm/models/_builder.py", line 418, in build_model_with_cfg load_pretrained( File "/.venv/lib/python3.11/site-packages/timm/models/_builder.py", line 215, in load_pretrained state_dict[weight_name] = adapt_input_conv(in_chans, state_dict[weight_name]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/timm/models/_manipulate.py", line 259, in adapt_input_conv O, I, J, K = conv_weight.shape ^^^^^^^^^^ ValueError: not enough values to unpack (expected 4, got 3)

— Reply to this email directly, view it on GitHubhttps://github.com/TorchDSP/torchsig/issues/239#issuecomment-2226541580, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AUN3HRQZU4GO52C7QYQNCTDZMBUF3AVCNFSM6AAAAABKZVB5ZCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMRWGU2DCNJYGA. You are receiving this because you were mentioned.Message ID: @.***>

pvallance commented 4 months ago

Sorry for the spam. This line:

File "/.venv/lib/python3.11/site-packages/torchsig/models/iq_models/efficientnet/efficientnet.py", line 276, in efficientnet_b4 timm.create_model(

will return a 1D model. You want to use models from torchsig/models/spectrogram_models . Those will be 2D.

Phil Vallance

RF Research Team

(240) 373-6289

From: Phillip Vallance Sent: Saturday, July 13, 2024 12:41:01 PM To: TorchDSP/torchsig; TorchDSP/torchsig Cc: Mention Subject: Re: [TorchDSP/torchsig] Can't load pretrained models anymore (Issue #239)

The default efficientnets from timm are 2D. We have utilities for converting these models to 1D (typically used for the narrowband datasets). It looks like you might be trying to load the weights of a 1D network into a 2D model. Could you pass along your script snippets that are using torchsig and/or the datasets?

Phil Vallance

RF Research Team

(240) 373-6289

From: Phillip Vallance Sent: Saturday, July 13, 2024 12:25:54 PM To: TorchDSP/torchsig; TorchDSP/torchsig Cc: Mention Subject: Re: [TorchDSP/torchsig] Can't load pretrained models anymore (Issue #239)

Nicholas,

Are you using the 1-D efficientnet from torchsig?

Phil Vallance

RF Research Team

(240) 373-6289

From: Nicholas Bruce @.***> Sent: Friday, July 12, 2024 7:55:09 PM To: TorchDSP/torchsig Cc: Phillip Vallance; Mention Subject: Re: [TorchDSP/torchsig] Can't load pretrained models anymore (Issue #239)

huggingface/pytorch-image-models#2150https://github.com/huggingface/pytorch-image-models/issues/2150 huggingface/pytorch-image-models#2215https://github.com/huggingface/pytorch-image-models/issues/2215 https://huggingface.co/timm/swinv2_tiny_window16_256.ms_in1k/discussions/1

These references got me one step closer (past the above error). I adjusted the timm.create_model() calls in efficientnet.py. I'm now getting the following (related) error:

Traceback (most recent call last): File "", line 198, in _run_module_as_main File "", line 88, in _run_code File "/train.py", line 135, in my_model = train_mymodel() ^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/click/core.py", line 1157, in call return self.main(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) ^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, ctx.params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke return __callback(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/train.py", line 127, in train_sigclr model = MyModel() File "/mymodel.py", line 28, in init self.encoder = Encoder() ^^^^^^^^^^^^^^^^^^^ File "/encoder.py", line 8, in init self.convnet = efficientnet_b4(pretrained=pretrained, path=path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/torchsig/models/iq_models/efficientnet/efficientnet.py", line 276, in efficientnet_b4 timm.create_model( File "/.venv/lib/python3.11/site-packages/timm/models/_factory.py", line 117, in create_model model = create_fn( ^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/timm/models/efficientnet.py", line 1913, in efficientnet_b4 model = _gen_efficientnet( ^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/timm/models/efficientnet.py", line 651, in _gen_efficientnet model = _create_effnet(variant, pretrained, model_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/timm/models/efficientnet.py", line 368, in _create_effnet model = build_model_with_cfg( ^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/timm/models/_builder.py", line 418, in build_model_with_cfg load_pretrained( File "/.venv/lib/python3.11/site-packages/timm/models/_builder.py", line 215, in load_pretrained state_dict[weight_name] = adapt_input_conv(in_chans, state_dict[weight_name]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/timm/models/_manipulate.py", line 259, in adapt_input_conv O, I, J, K = conv_weight.shape ^^^^^^^^^^ ValueError: not enough values to unpack (expected 4, got 3)

— Reply to this email directly, view it on GitHubhttps://github.com/TorchDSP/torchsig/issues/239#issuecomment-2226541580, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AUN3HRQZU4GO52C7QYQNCTDZMBUF3AVCNFSM6AAAAABKZVB5ZCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMRWGU2DCNJYGA. You are receiving this because you were mentioned.Message ID: @.***>

nsbruce commented 4 months ago

Hey thanks for the response @pvallance. This is based off of the narrow band example notebooks. In https://github.com/TorchDSP/torchsig/blob/main/examples/02_example_sig53_classifier.ipynb the model being used on the narrowband dataset is

model = efficientnet_b4(
    pretrained=True,
    path="efficientnet_b4.pt",
)

Which was working for me as an encoder while using torchsig v.0.5.0.

I'm using it like this:

from torchsig.models.iq_models.efficientnet.efficientnet import efficientnet_b4
import torch.nn as nn
import torch

class Encoder(nn.Module):
    def __init__(self, pretrained=True,path="/path/to/sig53/efficientnet_b4_online.pt",device='cuda'):
        super().__init__()
        self.convnet = efficientnet_b4(pretrained=pretrained, path=path)

        self.convnet = self.convnet.to(device)
    def forward(self, x):
        return self.convnet(x)

    def predict(self, x):
        with torch.no_grad():
            out = self.forward(x)
        return out

Which is causing that error when I try to build instantiate an Encoder().

pvallance commented 4 months ago

We are reworking these examples and updating our unit tests. I hope to have a new release in the next few days.

Phil Vallance

RF Research Team

(240) 373-6289

From: Nicholas Bruce @.***> Sent: Saturday, July 13, 2024 5:21:35 PM To: TorchDSP/torchsig Cc: Phillip Vallance; Mention Subject: Re: [TorchDSP/torchsig] Can't load pretrained models anymore (Issue #239)

Hey thanks for the response @pvallancehttps://github.com/pvallance. This is based off of the narrow band example notebooks. In https://github.com/TorchDSP/torchsig/blob/main/examples/02_example_sig53_classifier.ipynb the model being used on the narrowband dataset is

model = efficientnet_b4( pretrained=True, path="efficientnet_b4.pt", )

Which was working for me as an encoder while using torchsig v.0.5.0https://github.com/TorchDSP/torchsig/releases/tag/v0.5.0.

I'm using it like this:

from torchsig.models.iq_models.efficientnet.efficientnet import efficientnet_b4 import torch.nn as nn import torch

class Encoder(nn.Module): def init(self, pretrained=True,path="/path/to/sig53/efficientnet_b4_online.pt",device='cuda'): super().init() self.convnet = efficientnet_b4(pretrained=pretrained, path=path)

    self.convnet = self.convnet.to(device)
def forward(self, x):
    return self.convnet(x)

def predict(self, x):
    with torch.no_grad():
        out = self.forward(x)
    return out

Which is causing that error when I try to build instantiate an Encoder().

— Reply to this email directly, view it on GitHubhttps://github.com/TorchDSP/torchsig/issues/239#issuecomment-2227106247, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AUN3HRXBTJEJL3MHLN4OF6TZMGK57AVCNFSM6AAAAABKZVB5ZCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMRXGEYDMMRUG4. You are receiving this because you were mentioned.Message ID: @.***>

nsbruce commented 3 months ago

Hi @pvallance any update on this?

pvallance commented 3 months ago

Yes. We have updated all the notebooks and are finishing the unit tests. I am hopeful Monday of next week. Sorry, we have been refactoring a lot of code.

Phil Vallance

RF Research Team

(240) 373-6289

From: Nicholas Bruce @.***> Sent: Friday, August 9, 2024 12:20:14 PM To: TorchDSP/torchsig Cc: Phillip Vallance; Mention Subject: Re: [TorchDSP/torchsig] Can't load pretrained models anymore (Issue #239)

Hi @pvallancehttps://github.com/pvallance any update on this?

— Reply to this email directly, view it on GitHubhttps://github.com/TorchDSP/torchsig/issues/239#issuecomment-2278306320, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AUN3HRU4N6DTU7FQRUVIUCLZQTT35AVCNFSM6AAAAABKZVB5ZCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENZYGMYDMMZSGA. You are receiving this because you were mentioned.Message ID: @.***>

pvallance commented 3 months ago

Nicholas,

Sorry for the delay. The next release will happen Tuesday. After that we will be releasing foundational models as another baseline approach.

Phil Vallance

RF Research Team

(240) 373-6289

From: Phillip Vallance Sent: Friday, August 9, 2024 12:44:54 PM To: TorchDSP/torchsig; TorchDSP/torchsig Cc: Mention Subject: Re: [TorchDSP/torchsig] Can't load pretrained models anymore (Issue #239)

Yes. We have updated all the notebooks and are finishing the unit tests. I am hopeful Monday of next week. Sorry, we have been refactoring a lot of code.

Phil Vallance

RF Research Team

(240) 373-6289

From: Nicholas Bruce @.***> Sent: Friday, August 9, 2024 12:20:14 PM To: TorchDSP/torchsig Cc: Phillip Vallance; Mention Subject: Re: [TorchDSP/torchsig] Can't load pretrained models anymore (Issue #239)

Hi @pvallancehttps://github.com/pvallance any update on this?

— Reply to this email directly, view it on GitHubhttps://github.com/TorchDSP/torchsig/issues/239#issuecomment-2278306320, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AUN3HRU4N6DTU7FQRUVIUCLZQTT35AVCNFSM6AAAAABKZVB5ZCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENZYGMYDMMZSGA. You are receiving this because you were mentioned.Message ID: @.***>

acchione commented 3 months ago

This sounds very interesting. Any updates here? I was looking to pull the latest but I didn't want to dig through this repo if a big refactor is on the way.

nsbruce commented 2 months ago

Hi @pvallance, lots of interesting updates were released this week! Thank you. The newer notebooks no longer have an example of loading the pre-trained weights.

When I try

from torchsig.models import EfficientNet1d
import torch

mdl = EfficientNet1d(2,53,"b4")
mdl.load_state_dict(torch.load('efficientnet_b4_online.pt'))

I get the following error: 

RuntimeError: Error(s) in loading state_dict for EfficientNet:
        Unexpected key(s) in state_dict: "conv_stem.bias", "blocks.0.0.conv_dw.bias", "blocks.0.0.conv_pw.bias", "blocks.0.1.conv_dw.bias", "blocks.0.1.conv_pw.bias", "blocks.1.0.conv_pw.bias", "blocks.1.0.conv_dw.bias", "blocks.1.0.conv_pwl.bias", "blocks.1.1.conv_pw.bias", "blocks.1.1.conv_dw.bias", "blocks.1.1.conv_pwl.bias", "blocks.1.2.conv_pw.bias", "blocks.1.2.conv_dw.bias", "blocks.1.2.conv_pwl.bias", "blocks.1.3.conv_pw.bias", "blocks.1.3.conv_dw.bias", "blocks.1.3.conv_pwl.bias", "blocks.2.0.conv_pw.bias", "blocks.2.0.conv_dw.bias", "blocks.2.0.conv_pwl.bias", "blocks.2.1.conv_pw.bias", "blocks.2.1.conv_dw.bias", "blocks.2.1.conv_pwl.bias", "blocks.2.2.conv_pw.bias", "blocks.2.2.conv_dw.bias", "blocks.2.2.conv_pwl.bias", "blocks.2.3.conv_pw.bias", "blocks.2.3.conv_dw.bias", "blocks.2.3.conv_pwl.bias", "blocks.3.0.conv_pw.bias", "blocks.3.0.conv_dw.bias", "blocks.3.0.conv_pwl.bias", "blocks.3.1.conv_pw.bias", "blocks.3.1.conv_dw.bias", "blocks.3.1.conv_pwl.bias", "blocks.3.2.conv_pw.bias", "blocks.3.2.conv_dw.bias", "blocks.3.2.conv_pwl.bias", "blocks.3.3.conv_pw.bias", "blocks.3.3.conv_dw.bias", "blocks.3.3.conv_pwl.bias", "blocks.3.4.conv_pw.bias", "blocks.3.4.conv_dw.bias", "blocks.3.4.conv_pwl.bias", "blocks.3.5.conv_pw.bias", "blocks.3.5.conv_dw.bias", "blocks.3.5.conv_pwl.bias", "blocks.4.0.conv_pw.bias", "blocks.4.0.conv_dw.bias", "blocks.4.0.conv_pwl.bias", "blocks.4.1.conv_pw.bias", "blocks.4.1.conv_dw.bias", "blocks.4.1.conv_pwl.bias", "blocks.4.2.conv_pw.bias", "blocks.4.2.conv_dw.bias", "blocks.4.2.conv_pwl.bias", "blocks.4.3.conv_pw.bias", "blocks.4.3.conv_dw.bias", "blocks.4.3.conv_pwl.bias", "blocks.4.4.conv_pw.bias", "blocks.4.4.conv_dw.bias", "blocks.4.4.conv_pwl.bias", "blocks.4.5.conv_pw.bias", "blocks.4.5.conv_dw.bias", "blocks.4.5.conv_pwl.bias", "blocks.5.0.conv_pw.bias", "blocks.5.0.conv_dw.bias", "blocks.5.0.conv_pwl.bias", "blocks.5.1.conv_pw.bias", "blocks.5.1.conv_dw.bias", "blocks.5.1.conv_pwl.bias", "blocks.5.2.conv_pw.bias", "blocks.5.2.conv_dw.bias", "blocks.5.2.conv_pwl.bias", "blocks.5.3.conv_pw.bias", "blocks.5.3.conv_dw.bias", "blocks.5.3.conv_pwl.bias", "blocks.5.4.conv_pw.bias", "blocks.5.4.conv_dw.bias", "blocks.5.4.conv_pwl.bias", "blocks.5.5.conv_pw.bias", "blocks.5.5.conv_dw.bias", "blocks.5.5.conv_pwl.bias", "blocks.5.6.conv_pw.bias", "blocks.5.6.conv_dw.bias", "blocks.5.6.conv_pwl.bias", "blocks.5.7.conv_pw.bias", "blocks.5.7.conv_dw.bias", "blocks.5.7.conv_pwl.bias", "blocks.6.0.conv_pw.bias", "blocks.6.0.conv_dw.bias", "blocks.6.0.conv_pwl.bias", "blocks.6.1.conv_pw.bias", "blocks.6.1.conv_dw.bias", "blocks.6.1.conv_pwl.bias", "conv_head.bias".

Which is a much smaller number of unexpected keys than before (output was truncated in my fist comment on this issue), and no missing keys, which I think is good progress. Do you have any advice on how to load the pretrained weights?

ereoh commented 2 months ago

Hello @nsbruce,

Taking over for @pvallance, please look at examples/02_example_sig53_classifier.ipynb for more guidance.

nsbruce commented 2 months ago

Hi @ereoh, I did look through that. No pre-trained weights are loaded in that notebook.

In the old version the weights were loaded like so:

model = efficientnet_b4(
    pretrained=True,
    path="efficientnet_b4.pt",
)

In the new version the pre-trained weights are not used.

ereoh commented 2 months ago

Gotcha. Passing it on to developers -- we really appreciate your patience.

ereoh commented 2 weeks ago

Hello, just updating you that for this next release (v0.6.1) we will be updating our pretrained models with the correct weights for both Narrowband and Wideband. Thanks for your patience.