search

NVlabs / RADIO

Official repository for "AM-RADIO: Reduce All Domains Into One"
Other
636 stars 23 forks source link

larger-scale radio models #44

Open yeezhu opened 5 months ago

yeezhu commented 5 months ago

Are there any plans to train or release larger-scale models, such as those based on the ViT-G architecture?

gheinrich commented 5 months ago

Hello, thanks for asking! RADIO ViT-H/16 already outperforms bigger foundation models on most metrics we tested it on. Is there one downstream use case in particular you feel would benefit from a more expensive architecture?

yeezhu commented 5 months ago

I replaced EVA-CLIP-ViT-G with radio (ViT-H) in my VLM, the performance drops on general VQA tasks. So, I was wondering if there is a ViT-G version for radio.

mranzinger commented 5 months ago

Would you be able to provide a few more details on your setup? We haven't observed a reduction in metrics versus that model.