FileNotFoundError when trying to test config files in Conv3D

[tauhidtamim]

Greetings. Congratulations on your work. I was trying to reproduce your results in Google Colab using processed datasets and pretrained models. I stored 'test_frames' folder inside /Conv3D/data/ and tried to run "python Sign_Isolated_Conv3D_clip_test.py" inside /Conv3D/ folder. I ran into the following error, I'm pasting the error traceback.

---Traceback Starts----

######################Testing Started####################### Traceback (most recent call last): File "Sign_Isolated_Conv3D_clip_test.py", line 138, in val_loss = val_epoch(model, criterion, val_loader, device, 0, logger, writer, phase=phase, exp_name=exp_name) File "/content/CVPR21Chal-SLR/Conv3D/validation_clip.py", line 13, in val_epoch for batch_idx, data in enumerate(dataloader): File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/dataloader.py", line 517, in next data = self._next_data() File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/dataloader.py", line 1199, in _next_data return self._process_data(data) File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/dataloader.py", line 1225, in _process_data data.reraise() File "/usr/local/lib/python3.7/dist-packages/torch/_utils.py", line 429, in reraise raise self.exc_type(msg) FileNotFoundError: Caught FileNotFoundError in DataLoader worker process 0. Original Traceback (most recent call last): File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/_utils/worker.py", line 202, in _worker_loop data = fetcher.fetch(index) File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/_utils/fetch.py", line 44, in data = [self.dataset[idx] for idx in possibly_batched_index] File "/content/CVPR21Chal-SLR/Conv3D/dataset_sign_clip.py", line 111, in getitem images.append(self.read_images(selected_folder, i)) File "/content/CVPR21Chal-SLR/Conv3D/dataset_sign_clip.py", line 75, in read_images index_list = self.frame_indices_tranform_test(len(os.listdir(folder_path)), self.frames, clip_no) FileNotFoundError: [Errno 2] No such file or directory: '../data/test_frames/signer34_sample1'

----Traceback Ends----

However, there is a signer34_sample1 inside /Conv3D/data/test_frames/. Any insights will be appreciated. Thanks.

[jackyjsy]

Thanks for your interest and I am happy to help. I think the problem is in the data paths. You can try to modify Line32 and Line 33 in "Sign_Isolated_Conv3D_clip_test.py" to

data_path = "data/train_frames" data_path2 = "data/test_frames"

Hope that will solve your problem.

[tauhidtamim]

Thanks for the help! I was able to test fully. However, I have a confusion. After running "ensemble_multimodal_rgbd.py" on your provided .pkl files, I got the following output -

/content/CVPR21Chal-SLR/ensemble
100% 3742/3742 [00:00<00:00, 20832.23it/s]
3742
top1:  0.005077498663816141
top5:  0.021111704970603957

I got the same output after generating .pkl files on my own. Why is the accuracy only 0.5% & 2.1%? Shouldn't it be like 98%? As I am a beginner, maybe it's a misunderstanding on my part. So apologies for that. Note that I got the same result after generating .pkl files on my own.

[jackyjsy]

Thanks for the help! I was able to test fully. However, I have a confusion. After running "ensemble_multimodal_rgbd.py" on your provided .pkl files, I got the following output -

/content/CVPR21Chal-SLR/ensemble
100% 3742/3742 [00:00<00:00, 20832.23it/s]
3742
top1:  0.005077498663816141
top5:  0.021111704970603957

I got the same output after generating .pkl files on my own. Why is the accuracy only 0.5% & 2.1%? Shouldn't it be like 98%? As I am a beginner, maybe it's a misunderstanding on my part. So apologies for that. Note that I got the same result after generating .pkl files on my own.

The accuracy you obtained is normal. Since we don't have access to the test labels, the test label we are using is just a pseudo label (random integers). The accuracy of 98% was evaluated by uploading our predictions to the challenge server. If you train our model using train set and test our code on val set, it gives you correct calculation on accuracy.

[tauhidtamim]

Thanks for clarifying. Much appreciated.