search

qubvel-org / segmentation_models.pytorch

Semantic segmentation models with 500+ pretrained convolutional and transformer-based backbones.
https://smp.readthedocs.io/
MIT License
9.53k stars 1.66k forks source link

AttributeError: Can't pickle local object 'get_preprocessing_fn.<locals>._preprocess_input’ #12

Closed cocozfz closed 5 years ago

cocozfz commented 5 years ago

When I run "cars segmentation (camvid).ipynb", there was an error： AttributeError: Can't pickle local object 'get_preprocessing_fn.._preprocess_input' This error is in the "train model for 40 epochs" Thanks

qubvel commented 5 years ago

Hi, its look like you are trying to pickle preprocessing function (maybe it is a multiprocesiing in dataloader), what is your python version and how many workers is set in dataloader?

cocozfz commented 5 years ago

Hi, its look like you are trying to pickle preprocessing function (maybe it is a multiprocesiing in dataloader), what is your python version and how many workers is set in dataloader?

My python version is 3.6.5， and I just use the demo of yours (named cars segmentation (camvid).ipynb). And the num_workers=12 in train_loader, the num workers=4 in valid_loader.

Thanks for your reply！^_^

qubvel commented 5 years ago

What is your os? I test it with: Ubuntu 18.04 Python 3.6

And it works fine...

cocozfz commented 5 years ago

What is your os? I test it with: Ubuntu 18.04 Python 3.6

And it works fine...

My os is: Windows Server 2016 Python 3.6

qubvel commented 5 years ago

I guess it is somehow related to different multiprocessing behavior in windows and ubuntu

qubvel commented 5 years ago

Check is the fix works with updated package:

$ pip install -U git+https://github.com/qubvel/segmentation_models.pytorch@fix/pickling-preprocess-input

And let me know, I dont have a windows machine to check it by myself. Thanks.

cocozfz commented 5 years ago

Check is the fix works with updated package:

$ pip install -U git+https://github.com/qubvel/segmentation_models.pytorch@fix/pickling-preprocess-input

And let me know, I dont have a windows machine to check it by myself. Thanks.

I first used the pip instruction, which prompted an ERROR ：BrokenPipeError: [Errno 32] Broken pipe. And then I set numwork=0 in the dataloader, and it works! This problem is caused by multithreading under Windows and is related to the DataLoader class. the details here : https://github.com/pytorch/pytorch/pull/5585

Thank you very much! Have a good day^_^