search

openai / Video-Pre-Training

Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos
MIT License
1.24k stars 138 forks source link

Access to youtube data labelled by the IDM #42

Open roger-creus opened 2 months ago

roger-creus commented 2 months ago

Hi!

I am wondering if you released the YouTube dataset with the labels given by the IDM. If so, how could we access it?

Thx!

Miffyli commented 2 months ago

Hey. Unfortunately the original pretraining set of VPT was not released in any form. The only data that was released as part of VPT was the contractor data, and later the BASALT competition data, both of which are listed in the README.md.

brandonhoughton commented 2 months ago

You may already know this but MineDojo released a YouTube index if you just want a large collection of Minecraft gameplay!

roger-creus commented 2 months ago

Hey! Thanks for the info. The MineDojo dataset is nice but it's not action-labelled! It should be possible to use VPT's IDM to label it right?

Miffyli commented 2 months ago

@roger-creus Potentially yes: VPT's IDM has generally quite robust to different preprocessing, but might not be perfect. You could also try different techniques to make use of the data (e.g., like what MineDojo did with MineCLIP, or what follow-up papers have done).