AttributeError: Can't pickle local object 'get_cifar_data_loaders.<locals>.<lambda>'

[gbpollam]

• MemCNN version: newest
• PyTorch version: 1.12.1
• Python version: 3.9.16
• Operating System: Windows11

Description

I am trying to run train.py but I get the following error:

AttributeError: Can't pickle local object 'get_cifar_data_loaders.<locals>.<lambda>'

Do you have any idea why that happens?

Here's the complete error log (for clarity's sake: I named the conda environment "memcnn"):

Traceback (most recent call last):
  File "C:\Users\giova\Desktop\Implementazioni\memcnn\build\lib\memcnn\train.py", line 109, in <module>
    main(data_dir=Config()['data_dir'],
  File "C:\Users\giova\Desktop\Implementazioni\memcnn\build\lib\memcnn\train.py", line 87, in main
    run_experiment(
  File "C:\Users\giova\Desktop\Implementazioni\memcnn\build\lib\memcnn\train.py", line 48, in run_experiment
    trainer(manager, start_iter=last_iter, use_cuda=use_cuda, *args, **trainer_params)
  File "C:\Users\giova\Desktop\Implementazioni\memcnn\memcnn\trainers\classification.py", line 82, in train
    for ind, (x, label) in enumerate(train_loader):
  File "C:\Users\giova\miniconda3\envs\memcnn\lib\site-packages\torch\utils\data\dataloader.py", line 444, in __iter__
    return self._get_iterator()
  File "C:\Users\giova\miniconda3\envs\memcnn\lib\site-packages\torch\utils\data\dataloader.py", line 390, in _get_iterator
    return _MultiProcessingDataLoaderIter(self)
  File "C:\Users\giova\miniconda3\envs\memcnn\lib\site-packages\torch\utils\data\dataloader.py", line 1077, in __init__
    w.start()
  File "C:\Users\giova\miniconda3\envs\memcnn\lib\multiprocessing\process.py", line 121, in start
    self._popen = self._Popen(self)
  File "C:\Users\giova\miniconda3\envs\memcnn\lib\multiprocessing\context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Users\giova\miniconda3\envs\memcnn\lib\multiprocessing\context.py", line 327, in _Popen
    return Popen(process_obj)
  File "C:\Users\giova\miniconda3\envs\memcnn\lib\multiprocessing\popen_spawn_win32.py", line 93, in __init__
    reduction.dump(process_obj, to_child)
  File "C:\Users\giova\miniconda3\envs\memcnn\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'get_cifar_data_loaders.<locals>.<lambda>'

What I Did

After cloning and installing the repo through setup.py (and creating config.json and the necessary folders), I tried to run train.py by calling:

python -m memcnn.train revnet38 cifar10 --fresh

[silvandeleemput]

Hi, that's strange, I'll try testing this later today, using something close to your environment.

[silvandeleemput]

I can reproduce the problem under Windows 10 with Python 3.9.13 and PyTorch 2.0.1, It probably has something to do with newer versions of PyTorch and/or Python and/or a Windows environment. I'll try to figure it out and solve it tomorrow morning.

[gbpollam]

Hi, thank you very much!

[silvandeleemput]

Ok, this issue appears to be fixed, but GitHub is experiencing some hiccups and I cannot merge/create PRs yet. I hope they fix it in time so I can push the fix today, otherwise, I'll do it tomorrow.

[silvandeleemput]

I have released a new version 1.5.2 with the fix included, you should be able to upgrade memcnn:

pip install --upgrade memcnn