AM-RADIO 2.5 ViT-H ? - Githubissues

javiabellan commented 3 months ago

Congrats for solving the Mode Switching issue with radio 2.5 ! Now all the heads of the am-radio dragon breath fire at the same time! 🔥🔥🔥

Now I wonder what is the future of this project and if there is something I can help with.

is AM-RADIO 2.5 ViT-H planned on the roadmap ? Also I appreciate the inclusion of the sigLip model but there was any reason to drop the openai CLIP ?
including more models would be awesome, I think if the training code was released (and the approach for fighting model switch at different res) the community could include many awesome models heads (OCR, depth, instance seg, generative style transfer...)

mranzinger commented 3 months ago

Thanks Javi.

Yes, a new ViT-H is on the roadmap.

The reason that we switched to SigLIP instead of OpenAI CLIP is because we saw that it helped with metrics. Also because SigLIP has become quite popular in the VLLM domain due to it's across the board strong results, particularly for a ViT-L model.

We're currently in the process of writing up the things that have changed that allowed us to produce the new v2.5 models, which also outlines how we addressed mode switching. Certainly the tech report isn't a sufficient explanation 😅 Among other things, the roadmap for this project does still include publicly releasing the training code.

javiabellan commented 3 months ago

Awesome! Thanks for sharing the roadmap.

mranzinger commented 3 weeks ago

Hi @javiabellan , sorry, forgot to update you that RADIOv2.5-H is now released 😃

javiabellan commented 3 weeks ago

Awesome! Great work!

NVlabs / RADIO

AM-RADIO 2.5 ViT-H ? #78