rom1504 / clip-retrieval

Easily compute clip embeddings and build a clip retrieval system with them
https://rom1504.github.io/clip-retrieval/
MIT License
2.42k stars 213 forks source link

Failed to reproduce the notebook example #185

Closed jpzhangvincent closed 10 months ago

jpzhangvincent commented 2 years ago

I followed the installation instruction and this notebook example on my Mac M1 max laptop. But I failed at running clip-retrieval inference --input_dataset image_folder --output_folder embedding_folder(even with the absolute path)

Here is the error trace

The number of samples has been estimated to be 3
Traceback (most recent call last):
  File "/opt/homebrew/Caskroom/miniforge/base/envs/nft-copycat-clip/bin/clip-retrieval", line 8, in <module>
    sys.exit(main())
  File "/opt/homebrew/Caskroom/miniforge/base/envs/nft-copycat-clip/lib/python3.9/site-packages/clip_retrieval/cli.py", line 16, in main
    fire.Fire(
  File "/opt/homebrew/Caskroom/miniforge/base/envs/nft-copycat-clip/lib/python3.9/site-packages/fire/core.py", line 141, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/nft-copycat-clip/lib/python3.9/site-packages/fire/core.py", line 466, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
  File "/opt/homebrew/Caskroom/miniforge/base/envs/nft-copycat-clip/lib/python3.9/site-packages/fire/core.py", line 681, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/nft-copycat-clip/lib/python3.9/site-packages/clip_retrieval/clip_inference/main.py", line 144, in main
    distributor()
  File "/opt/homebrew/Caskroom/miniforge/base/envs/nft-copycat-clip/lib/python3.9/site-packages/clip_retrieval/clip_inference/distributor.py", line 13, in __call__
    self.runner(i)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/nft-copycat-clip/lib/python3.9/site-packages/clip_retrieval/clip_inference/runner.py", line 36, in __call__
    batch = iterator.__next__()
  File "/opt/homebrew/Caskroom/miniforge/base/envs/nft-copycat-clip/lib/python3.9/site-packages/clip_retrieval/clip_inference/reader.py", line 207, in __iter__
    for batch in self.dataloader:
  File "/opt/homebrew/Caskroom/miniforge/base/envs/nft-copycat-clip/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 444, in __iter__
    return self._get_iterator()
  File "/opt/homebrew/Caskroom/miniforge/base/envs/nft-copycat-clip/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 390, in _get_iterator
    return _MultiProcessingDataLoaderIter(self)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/nft-copycat-clip/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1077, in __init__
    w.start()
  File "/opt/homebrew/Caskroom/miniforge/base/envs/nft-copycat-clip/lib/python3.9/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/nft-copycat-clip/lib/python3.9/multiprocessing/context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/nft-copycat-clip/lib/python3.9/multiprocessing/context.py", line 284, in _Popen
    return Popen(process_obj)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/nft-copycat-clip/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/nft-copycat-clip/lib/python3.9/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/nft-copycat-clip/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 47, in _launch
    reduction.dump(process_obj, fp)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/nft-copycat-clip/lib/python3.9/multiprocessing/reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'get_image_dataset.<locals>.ImageDataset'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/opt/homebrew/Caskroom/miniforge/base/envs/nft-copycat-clip/lib/python3.9/multiprocessing/spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/nft-copycat-clip/lib/python3.9/multiprocessing/spawn.py", line 126, in _main
    self = reduction.pickle.load(from_parent)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/nft-copycat-clip/lib/python3.9/multiprocessing/synchronize.py", line 110, in __setstate__
    self._semlock = _multiprocessing.SemLock._rebuild(*state)
FileNotFoundError: [Errno 2] No such file or directory
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/opt/homebrew/Caskroom/miniforge/base/envs/nft-copycat-clip/lib/python3.9/multiprocessing/spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/nft-copycat-clip/lib/python3.9/multiprocessing/spawn.py", line 126, in _main
    self = reduction.pickle.load(from_parent)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/nft-copycat-clip/lib/python3.9/multiprocessing/synchronize.py", line 110, in __setstate__
    self._semlock = _multiprocessing.SemLock._rebuild(*state)
FileNotFoundError: [Errno 2] No such file or directory
BarNehemia commented 2 years ago

@jpzhangvincent did you managed? I'm getting this error too.

rom1504 commented 10 months ago

should be fixed now