Open pmeier opened 2 years ago
Can I take this up?
Sure, go ahead.
The Places365-Challenge dataset is way too big for me to download and find the sha256 code, is there any other way to find it?
I think I habe the files in disk. Send the PR without it. I'll fill them in.
I'm sorry but I just realized I will need to download the entire dataset to build the data pipe and I cannot do that for such a huge dataset. Please do unassign this issue, I will look into the other datasets that I will be able to download and try to go forward with them.
Regret any inconvenience caused.
Should I make a PR with resources that I have updated or should I just let it be?
I'm sorry but I just realized I will need to download the entire dataset to build the data pipe and I cannot do that for such a huge dataset. Please do unassign this issue, I will look into the other datasets that I will be able to download and try to go forward with them.
Regret any inconvenience caused.
No worries, I could have told you before you got started and this is on me. Still, before I unassign you, would you be willing to implement this for the small variant of the dataset. This limits the download size for all files to about 30GB. IIRC, they have exactly the same structure as their larger "sisters". So the only thing I would have to do after your PR is checking if the implementation still works with the other files.
Should I make a PR with resources that I have updated or should I just let it be?
Depends. Do you have more than just the skeleton? If yes, go ahead and send the PR. Otherwise I don't think it will be of much help for someone taking over.
I only have 25GB on the drive which has PyTorch so I won't be able to run it on my device sadly. I have added all the links for the resource download, image type, and all the other links so I'll add a PR as it'll help start off whoever picks up the issue next.
In the meanwhile, I have looked into the RenderedSST2 and I would be able to take that up. Is there any quirk to that dataset I should be wary of before commenting on the issue?
In the meanwhile, I have looked into the RenderedSST2 and I would be able to take that up. Is there any quirk to that dataset I should be wary of before commenting on the issue?
No, I don't think so. Go ahead, I can assign you there.
For anyone who wants to pick this up: you can find the skeleton for the implementation in #5383.
cc @pmeier @bjuncek