MineDojo / MineCLIP

Foundation Model for MineDojo
MIT License
226 stars 30 forks source link

Data preprocess details of MineCLIP #16

Open Ray7788 opened 5 months ago

Ray7788 commented 5 months ago

Hi there, First of all, thank you for your amazing work!

Recently I have been fine-tuning this pre-trained weight(attn.pth), I have prepared some video clips(mp4 files: 16 consecutive frames with 160 × 256 resolution) with matched text(JSON file formatted) for my fine-tuning task. I tried to preprocess my dataset based on what you mentioned in your paper, but I am a little confused about how you preprocessed raw video clips before as the input of MineCLIP, Did you use OpenCV or Imageio, etc. thank you a lot!

Sorry I might be stupid enough to ask this question as I am a new beginner in this field, thank you for your time and patience!