ValueError: num_samples should be a positive integer value, but got num_samples=0

[yiheo]

Hi, again. :) I'm still in python train_fine.py -gpu {gpu_num}

dataset size:0 Traceback (most recent call last): File "train_fine.py", line 269, in run() File "train_fine.py", line 86, in run dataloader = torch.utils.data.DataLoader(dataset, batch_size=batch_size, shuffle=True, File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 262, in __init__ sampler = RandomSampler(dataset, generator=generator) # type: ignore File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/sampler.py", line 103, in __init__ raise ValueError("num_samples should be a positive integer " ValueError: num_samples should be a positive integer value, but got num_samples=0

I downloaded Charades dataset(Charades_v1_rgb.tar 76GB) and unzipped it. However, dataset size is still zero.

What can I do?

+) Plus, extract_fineFEAT.py also has import videotransforms.

[kkahatapitiya]

If you have all the paths correct in the training script, Please look at the following line. If you run the script with a wrong/empty path at the first try, if would create this empty labeldata file. You have to delete this file and re-run with the paths fixed in the training script. It will create the proper labeldata file at your first run, so that it would be faster to initialize the training script in later runs.

https://github.com/kkahatapitiya/Coarse-Fine-Networks/blob/b2e71df4c9b51b9ad8152b4c4c17ce5770417d13/charades_fine.py#L92

+) Plus, extract_fineFEAT.py also has import videotransforms

Thank you for pointing that out, I will comment this line in the next commit.

[yiheo]

[edited] Thanks for your comments. You've been very helpful. I really appreciate it. My problem was https://github.com/kkahatapitiya/Coarse-Fine-Networks/blob/b2e71df4c9b51b9ad8152b4c4c17ce5770417d13/train_fine.py#L52 After I modified '/data/.../Charades_v1_rgb' into 'data/.../Charades_v1_rgb' (delete the first slash), it works with nonzero samples.

Then, I faced to several errors. And I solved them as below.

1. ValueError: too many values to unpack (expected 2) in https://github.com/kkahatapitiya/Coarse-Fine-Networks/blob/b2e71df4c9b51b9ad8152b4c4c17ce5770417d13/charades_fine.py#L162 When I checked the return value of load_rgb_frames, the format is a list.

   [<PIL.Image.Image image mode=RGB size=320x240 at 0x7FF5D409E4F0>, <PIL.Image.Image image mode=RGB size=320x240 at 0x7FF5D405BDF0>, <PIL.Image.Image image mode=RGB size=320x240 at 0x7FF5D4064400>, <PIL.Image.Image image mode=RGB size=320x240 at 0x7FF5D4064550>, <PIL.Image.Image image mode=RGB size=320x240 at 0x7FF5D4064490>, <PIL.Image.Image image mode=RGB size=320x240 at 0x7FF5D4064610>, <PIL.Image.Image image mode=RGB size=320x240 at 0x7FF5D40646A0>, <PIL.Image.Image image mode=RGB size=320x240 at 0x7FF5D4064730>, <PIL.Image.Image image mode=RGB size=320x240 at 0x7FF5D40647C0>, <PIL.Image.Image image mode=RGB size=320x240 at 0x7FF5D4064850>, <PIL.Image.Image image mode=RGB size=320x240 at 0x7FF5D40648E0>, <PIL.Image.Image image mode=RGB size=320x240 at 0x7FF5D4064970>]

To remove the value error, I just edit the line into imgs = load_rgb_frames(self.root, vid, start_f, frames, stride_f, self.loader) #stride_f without feat. (But, I cannot sure that what is your intentions...)

2. AttributeError: 'Tensor' object has no attribute 'keys' in https://github.com/kkahatapitiya/Coarse-Fine-Networks/blob/b2e71df4c9b51b9ad8152b4c4c17ce5770417d13/charades_fine.py#L213 So, I just comments the line.

3. ValueError: too many values to unpack (expected 4) in https://github.com/kkahatapitiya/Coarse-Fine-Networks/blob/b2e71df4c9b51b9ad8152b4c4c17ce5770417d13/train_fine.py#L178 When I checked data list, data[3] is

   tensor([[ 0, 36, 36, 1], [14, 64, 93, 1], [11, 64, 77, 1], [ 5, 64, 74, 1], [11, 64, 82, 1], [ 0, 53, 53, 1], [ 2, 64, 66, 1], [ 3, 64, 67, 1]])

I guess it is batch or something. So, I edit the code from inputs, labels, masks, name = data into inputs, labels, masks, _, name = data.

Please check what I did and reply comments. Thanks.

[yeboqxc]

If you have all the paths correct in the training script, Please look at the following line. If you run the script with a wrong/empty path at the first try, if would create this empty labeldata file. You have to delete this file and re-run with the paths fixed in the training script. It will create the proper labeldata file at your first run, so that it would be faster to initialize the training script in later runs.

https://github.com/kkahatapitiya/Coarse-Fine-Networks/blob/b2e71df4c9b51b9ad8152b4c4c17ce5770417d13/charades_fine.py#L92

+) Plus, extract_fineFEAT.py also has import videotransforms

Thank you for pointing that out, I will comment this line in the next commit.

Hello, I did as you said, delete labeldata_160.npy, and run python train_fine.py -gpu {1}, but still this bug: ValueError: num_samples should be a positive integer value, but got num_samples=0

ok!I solved this bug by changing shuffle=True to shuffle=False

[yiheo]

If you have all the paths correct in the training script, Please look at the following line. If you run the script with a wrong/empty path at the first try, if would create this empty labeldata file. You have to delete this file and re-run with the paths fixed in the training script. It will create the proper labeldata file at your first run, so that it would be faster to initialize the training script in later runs. https://github.com/kkahatapitiya/Coarse-Fine-Networks/blob/b2e71df4c9b51b9ad8152b4c4c17ce5770417d13/charades_fine.py#L92

+) Plus, extract_fineFEAT.py also has import videotransforms

Thank you for pointing that out, I will comment this line in the next commit.

Hello, I did as you said, delete labeldata_160.npy, and run python train_fine.py -gpu {1}, but still this bug: ValueError: num_samples should be a positive integer value, but got num_samples=0

ok!I solved this bug by changing shuffle=True to shuffle=False

After I modified '/data/.../Charades_v1_rgb' into 'data/.../Charades_v1_rgb' (delete the first slash), it works with nonzero samples.

[kkahatapitiya]

After I modified '/data/.../Charades_v1_rgb' into 'data/.../Charades_v1_rgb' (delete the first slash), it works with nonzero samples.

My dataset was in the path '/data/.../Charades_v1_rgb'. As long as you set the path correct, that should be fine.

Then, I faced to several errors. And I solved them as below.

Yes, I see these issues now. Sorry about them, I will fix them. Having similar implementations for charades_fine.py and charades_coarse_fineFEAT.py may have made me oversee these issues.

1. ValueError: too many values to unpack (expected 2) in https://github.com/kkahatapitiya/Coarse-Fine-Networks/blob/b2e71df4c9b51b9ad8152b4c4c17ce5770417d13/charades_fine.py#L162 To remove the value error, I just edit the line into imgs = load_rgb_frames(self.root, vid, start_f, frames, stride_f, self.loader) #stride_f without feat. (But, I cannot sure that what is your intentions...)

feat is returned in charades_coarse_fineFEAT.py only, and should not be in this file.

1. AttributeError: 'Tensor' object has no attribute 'keys' in https://github.com/kkahatapitiya/Coarse-Fine-Networks/blob/b2e71df4c9b51b9ad8152b4c4c17ce5770417d13/charades_fine.py#L213 So, I just comments the line. Yes, keys is not used in this file

2. ValueError: too many values to unpack (expected 4) in https://github.com/kkahatapitiya/Coarse-Fine-Networks/blob/b2e71df4c9b51b9ad8152b4c4c17ce5770417d13/train_fine.py#L178 I guess it is batch or something. So, I edit the code from inputs, labels, masks, name = data into inputs, labels, masks, _, name = data.

This is some meta information not used in the final version of the code. Will fix these.

Thanks for pointing them out, appreciate it!