Downloading the kinetics dataset

[sahilg06]

🐛 Describe the bug

Not able to download the kinetics dataset through torchvision.datasets.Kinetics class.

I used the following command ds = torchvision.datasets.Kinetics(root='ktest',frames_per_clip=100, num_classes='600', split='val', download=True, num_download_workers=1000, num_workers=2)

it shows the following error after sometime

I was able to get rid of this error by finding and replacing spaces with %20 in the k600_val_path.txt But still few URLs were not working.

Versions

cc @pmeier

[sahilg06]

@pmeier @bjuncek I think we can add line = str(line).replace(" ", "%20") after this line to get rid of the above bug. And we can also do something similar here May I initiate a pull request doing that? Thanks!

[sahilg06]

Also, according to the code of _make_ds_structure function This url should be of CSV file containing all the details but instead its a txt file, which produces error while using the Kinetcis class. Here is the error

[bjuncek]

Hi @sahilg06 -- the above URL fix seams reasonable to me. If you can send a PR with the fix, I can review it and test it out on Friday (I'm off due to ECCV :)).

Regarding the .csv vs .txt, I don't know how this issue creeped in, but if I'm not mistaken this should just be changed to .csv? (at least that works for me)

If that's the case, feel free to just send everything in a single PR and I'll take a look at it tomorrow or on Friday. Thanks!!!

[sahilg06]

Hi @bjuncek, Sure, I will send the PR. But for the 2nd issue(.csv vs .txt), https://s3.amazonaws.com/kinetics/600/annotations/train.csv and https://s3.amazonaws.com/kinetics/600/annotations/val.csv don't work, perhaps just changing .txt with .csv will not resolve the issue.

[bjuncek]

So train and val are .txt but the test is .csv

I’ll try to send a PR to upstream CVDF to see if we can change that — otherwise we can just have an annoying special case for test I guess.

[sahilg06]

Actually, I didn't know that split argument supports 'test' before you told me because in the docs its mentioned that split supports only train and val. So I guess 'test' should also be added there.

And according to the code they expect "youtube_id", "time_start", "time_end" in the .csv file. But even if for the train and val dataset there is a .txt file, that file doesn't contain enough info (like youtube id, time_start, time_end, etc) to make that code work for the Kinetics600 dataset.

see, train.txt for kinetics600 looks like this

So according to me, for Kinetics400, everything is good. and For Kinetics600, it will only work for test data. Train and val files should be completely changed. Thanks!

[sahilg06]

@bjuncek May you please review the above PR and merge it?

[sahilg06]

@bjuncek these are the required files for kinetics 600 I guess train.csv validate.csv

[bjuncek]

@sahilg06 done - approved, I just need someone with higher powers to merge it. Thanks a lot

[bjuncek]

As for the K600 files, I'm double checking with Joao what's the deal about the different file format. Thanks for generating these -- we might need to use them after all

[pmeier]

Just to clear some confusion for someone else reading this thread: When @bjuncek talks about "Joao", he doesn't mean @jdsgomes, but rather Joao Carreira operating under @kinetics-cvdf. The links were updated in https://github.com/cvdfoundation/kinetics-dataset/commit/48a523ea10ad9d286e4f6f2895910fc4737e1ac5.