ViT-tiny encoder - Githubissues

facebookresearch / jepa

PyTorch code and models for V-JEPA self-supervised learning from video.

Other

2.6k stars 248 forks source link

ViT-tiny encoder #22

Open jez-moxmo opened 6 months ago

jez-moxmo commented 6 months ago

Hi, I can see there are various encoder class configs including a ViT-mini. Will you be releasing one in the future?

vedal commented 5 months ago

It seems to be missing from ijepa as well, with the following issue which hasnt been responded to. Any comments on whether any of these configs will be released @MidoAssran ?

rolson24 commented 4 months ago

I'm guessing that it just doesn't learn very much. Working on a project using JEPA to train some models, so will try it and let you know.