Request a complete requirements.txt file

[songquanpeng]

Hello, when I try to run your code with the following argument: --dataset miniimagenet --mode train, a strange error occurred:

D:\Software\Anaconda3\envs\pytorch\lib\site-packages\torch\optim\lr_scheduler.py:82: UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`. In PyTorch 1.1.0 and later, you should call them in the opposite order: `optimizer.step()` before `lr_scheduler.step()`.  Failure to do this will result in PyTorch skipping the first value of the learning rate schedule.See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate
  "https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate", UserWarning)
  0% 0/541 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "C:\Program Files\JetBrains\PyCharm 2020.1.2\plugins\python\helpers\pydev\_pydev_bundle\pydev_umd.py", line 197, in runfile
    pydev_imports.execfile(filename, global_vars, local_vars)  # execute the script
  File "C:\Program Files\JetBrains\PyCharm 2020.1.2\plugins\python\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "D:/Research/Code/ICI-FSL/main.py", line 174, in <module>
    main(args)
  File "D:/Research/Code/ICI-FSL/main.py", line 164, in main
    train_embedding(args)
  File "D:/Research/Code/ICI-FSL/main.py", line 60, in train_embedding
    for images, labels in tqdm(source_loader, ncols=0):
  File "D:\Software\Anaconda3\envs\pytorch\lib\site-packages\tqdm\std.py", line 1130, in __iter__
    for obj in iterable:
  File "D:\Software\Anaconda3\envs\pytorch\lib\site-packages\torch\utils\data\dataloader.py", line 278, in __iter__
    return _MultiProcessingDataLoaderIter(self)
  File "D:\Software\Anaconda3\envs\pytorch\lib\site-packages\torch\utils\data\dataloader.py", line 682, in __init__
    w.start()
  File "D:\Software\Anaconda3\envs\pytorch\lib\multiprocessing\process.py", line 112, in start
    self._popen = self._Popen(self)
  File "D:\Software\Anaconda3\envs\pytorch\lib\multiprocessing\context.py", line 223, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "D:\Software\Anaconda3\envs\pytorch\lib\multiprocessing\context.py", line 322, in _Popen
    return Popen(process_obj)
  File "D:\Software\Anaconda3\envs\pytorch\lib\multiprocessing\popen_spawn_win32.py", line 89, in __init__
    reduction.dump(process_obj, to_child)
  File "D:\Software\Anaconda3\envs\pytorch\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <function <lambda> at 0x000001489FB634C8>: attribute lookup <lambda> on datasets failed
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "D:\Software\Anaconda3\envs\pytorch\lib\multiprocessing\spawn.py", line 105, in spawn_main
    exitcode = _main(fd)
  File "D:\Software\Anaconda3\envs\pytorch\lib\multiprocessing\spawn.py", line 115, in _main
    self = reduction.pickle.load(from_parent)
EOFError: Ran out of input

It looks like something wrong with the data load part. So I want to make sure all the packages such as multiprocessing have the proper version to help locate the error.

Would you please offer a requirements.txt exported by pip freeze > requirements.txt? Thanks in advance.

[songquanpeng]

Looks like it helps if you set num_workders=0 when you call the DataLoader function. So I think it's a problem caused by multiprocessing.

    def __iter__(self):
        if self.num_workers == 0:
            return _SingleProcessDataLoaderIter(self)
        else:
            return _MultiProcessingDataLoaderIter(self)

May I ask which operating system do you use in this experiment?

[Yikai-Wang]

Our experiments are running on Ubuntu 16.04.5 LTS (GNU/Linux 4.4.0-87-generic x86_64). Based on your description, I am more inclined that this is an operating system related issue rather than package issue. And the multiprocessing seems not a package but a file in pytorch. The requirement file is here. Hope it can help you.

[songquanpeng]

Thanks for your help. Looks like multiprocessing is a standard package of Python. And my env name is pytorch which may confuse you.