search

ruotianluo / DiscCaptioning

Code for Discriminability objective for training descriptive captions(CVPR 2018)
110 stars 21 forks source link

FileNotFoundError: [Errno 2] No such file or directory: 'cider/data/cocotalk_fc\\391895.npy Terminating BlobFetcher #22

Closed Xtraordinaire9911 closed 1 year ago

Xtraordinaire9911 commented 1 year ago

After using a whole afternoon to fix 87 bugs in this repo one by one, finally I'm able to start training, but not surprisingly, I got stuck by another issue. This time I have no more idea about how to fix it, as there exists barely any related Q&A on Google (e.g. What is the 391895.npy? Where to find it? What may replace it? Or how to bypass this appear-to-be-easy FileNotFoundError?). Would anyone kindly give a hint of how I might sort out this issue? Many thanks.

The code to trigger this issue (after following all the previous steps): bash run_fc_con.sh The error message:

DataLoader loading json file: data/cocotalk.json vocab size is 9487 DataLoader loading h5 file: data/cocotalk_fc data/cocobu_att data/cocotalk_label.h5 max sequence length in data is 16 read 123287 image features assigned 113287 images to split train assigned 5000 images to split val assigned 5000 images to split test

Traceback (most recent call last): File "train.py", line 242, in train(opt) File "train.py", line 109, in train data = loader.get_batch('train') File "D:\Project\DiscCaptioning\dataloader.py", line 138, in get_batch ix, tmp_wrapped = self._prefetch_process[split].get() File "D:\Project\DiscCaptioning\dataloader.py", line 264, in get tmp = self.split_loader.next() File "D:\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 521, in next data = self._next_data() File "D:\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 561, in _next_data data = self._dataset_fetcher.fetch(index) # may raise StopIteration File "D:\Anaconda3\lib\site-packages\torch\utils\data_utils\fetch.py", line 44, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "D:\Anaconda3\lib\site-packages\torch\utils\data_utils\fetch.py", line 44, in data = [self.dataset[idx] for idx in possibly_batched_index] File "D:\Project\DiscCaptioning\dataloader.py", line 205, in getitem return (np.load(os.path.join("cider/" + self.input_fc_dir, str(self.info['images'][ix]['id']) + '.npy')), np.zeros((1,1)), ix) File "D:\Anaconda3\lib\site-packages\numpy\lib\npyio.py", line 428, in load fid = open(os_fspath(file), "rb") FileNotFoundError: [Errno 2] No such file or directory: 'cider/data/cocotalk_fc\\391895.npy' Terminating BlobFetcher

(After downloading and unzipping the cocotalk_fc.tar from Google drive, I obtain a folder cocotalk_fc with structure like this: cocotalk_fc (the root folder obtained from zipping) -> cocotalk_fc (a binary file) and after I cut them to under the directory cider, the path to the end of this directory branch is like this: DiscCaptioning -> cider -> cocotalk_fc (the root folder after zipping) -> cocotalk_fc (a binary file) Also, it won't help if you simply rename the binary cocotalk_fc file to 391895 or 391895.npy, which would still throw the same error. I'm hence stuck.)

ruotianluo commented 1 year ago

You should get a list of npy after untar.

Xtraordinaire9911 commented 1 year ago

You should get a list of npy after untar.

Thank you very much for your quick response. I encountered the issue by misusing zipping app for untar, and the issue is now solved by ruuning the following command (win10 cmd): tar -xvzf [\PATH\TO\FILE\]cocotalk_fc.tar -C [\PATH\TO\FOLDER\EXTRACTION]

As a result, after untar I obtain a folder named cocotalk_fc containing 123,287 npy files indexing up to 581929.