ValueError: source code string cannot contain null bytes

[annahambi]

I am trying to convert a VGG16 model (loaded from the PyTorch models, however modified to be trained on CIFAR10) to an SNN. When I run the conversion, the import script generates a "ValueError: source code string cannot contain null bytes".

Full Error Message

Initializing INI simulator...

Loading data set from '.npz' files in /home/annah/TII_team Dropbox/Cryptography Research Centre/ML Team/Experiments_SNN_Toolbox/02b_data.

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-9-693d1b398812> in <module>
      4 ###################
      5 
----> 6 main(config_filepath)

~/miniconda3/envs/tf20201215b/lib/python3.8/site-packages/snntoolbox/bin/run.py in main(filepath)
     29     if filepath is not None:
     30         config = update_setup(filepath)
---> 31         run_pipeline(config)
     32         return
     33 

~/miniconda3/envs/tf20201215b/lib/python3.8/site-packages/snntoolbox/bin/utils.py in run_pipeline(config, queue)
     70                                   config.get('input', 'model_lib') +
     71                                   '_input_lib')
---> 72         input_model = model_lib.load(config.get('paths', 'path_wd'),
     73                                      config.get('paths', 'filename_ann'))
     74 

~/miniconda3/envs/tf20201215b/lib/python3.8/site-packages/snntoolbox/parsing/model_libs/pytorch_input_lib.py in load(path, filename)
     56 
     57     # Load the Pytorch model.
---> 58     mod = import_script(path, filename)
     59     kwargs = mod.kwargs if hasattr(mod, 'kwargs') else {}
     60     model_pytorch = mod.Model(**kwargs)

~/miniconda3/envs/tf20201215b/lib/python3.8/site-packages/snntoolbox/utils/utils.py in import_script(path, filename)
    188         spec = importlib.util.spec_from_file_location(filename, filepath)
    189         mod = importlib.util.module_from_spec(spec)
--> 190         spec.loader.exec_module(mod)
    191     elif v >= (3, 3):
    192         # noinspection PyCompatibility,PyUnresolvedReferences

~/miniconda3/envs/tf20201215b/lib/python3.8/importlib/_bootstrap_external.py in exec_module(self, module)

~/miniconda3/envs/tf20201215b/lib/python3.8/importlib/_bootstrap_external.py in get_code(self, fullname)

~/miniconda3/envs/tf20201215b/lib/python3.8/importlib/_bootstrap_external.py in source_to_code(self, data, path, _optimize)

~/miniconda3/envs/tf20201215b/lib/python3.8/importlib/_bootstrap.py in _call_with_frames_removed(f, *args, **kwds)

ValueError: source code string cannot contain null bytes

Generate VGG16 via

# Create pytorch model from PyTorch with a modified input size.
model = Custom_VGG(ipt_size=(32, 32), 
                   pretrained=False, 
                   vgg_type=model_name,
                    num_classes=10)

Custom class

"""
The following code is based on the following StackOverflow answer

https://stackoverflow.com/questions/46963372/a-proper-way-to-adjust-input-size-of-cnn-e-g-vgg 
"""

import torch
from torch.autograd import Variable
import torchvision
import torch.nn as nn

from torchvision.models.vgg import model_urls

VGG_TYPES = {'vgg11' : torchvision.models.vgg11, 
             'vgg11_bn' : torchvision.models.vgg11_bn, 
             'vgg13' : torchvision.models.vgg13, 
             'vgg13_bn' : torchvision.models.vgg13_bn, 
             'vgg16' : torchvision.models.vgg16, 
             'vgg16_bn' : torchvision.models.vgg16_bn,
             'vgg19_bn' : torchvision.models.vgg19_bn, 
             'vgg19' : torchvision.models.vgg19}

class Custom_VGG(nn.Module):

    def __init__(self,
                 ipt_size=(128, 128), 
                 pretrained=True, 
                 vgg_type='vgg19_bn', 
                 num_classes=1000):
        super(Custom_VGG, self).__init__()

        # load convolutional part of vgg
        assert vgg_type in VGG_TYPES, "Unknown vgg_type '{}'".format(vgg_type)
        vgg_loader = VGG_TYPES[vgg_type]
        vgg = vgg_loader(pretrained=pretrained)
        self.features = vgg.features

        # init fully connected part of vgg
        test_ipt = Variable(torch.zeros(1,3,ipt_size[0],ipt_size[1]))
        test_out = vgg.features(test_ipt)
        self.n_features = test_out.size(1) * test_out.size(2) * test_out.size(3)
        self.classifier = nn.Sequential(nn.Linear(self.n_features, 4096),
                                        nn.ReLU(True),
                                        nn.Dropout(),
                                        nn.Linear(4096, 4096),
                                        nn.ReLU(True),
                                        nn.Dropout(),
                                        nn.Linear(4096, num_classes)
                                       )
        self._init_classifier_weights()

    def forward(self, x):
        x = self.features(x)
        x = x.view(x.size(0), -1)
        x = self.classifier(x)
        return x

    def _init_classifier_weights(self):
        for m in self.classifier:
            if isinstance(m, nn.Linear):
                m.weight.data.normal_(0, 0.01)
                m.bias.data.zero_()

[rbodo]

You might have to name your class "Model" instead of "Custom_VGG", but I think your issue is unrelated to that. Seems to be a parsing issue when using the mod = importlib.util.module_from_spec(spec) function. Perhaps inspecting the mod object will help see if there is something in your script that cannot be properly parsed.

[annahambi]

I have done as you suggested and renamed my class to "Model" instead of "Custom_VGG". As you suspected already, the error message stayed the same. Can you give me a hint on how I can inspect the mod object? I have no idea how such an inspection works : P

[rbodo]

If you are using an IDE like PyCharm or Spyder, you can use the debug function to set a breakpoint and look at the contents of mod. Otherwise you can insert a print statement, for instance print(mod.__dict__). I won't be able to further help much with the debugging though. If all fails you might want to consider setting up your network in Keras instead of Pytorch to avoid this error. You could test with dummy weights first if the parsing goes through using Keras.