Interest in implementing pre-trained weights from "ImageNet-21K Pretraining for the Masses" to vit-keras

[johnypark]

Hello,

Thank you so much for this great repo. Everything works beautifully so far.

I was wondering if you guys are interested in porting pytorch ViT model from a 2021 NeurlIPS paper "ImageNet-21K Pretraining for the Masses"(https://github.com/Alibaba-MIIL/ImageNet21K), which curated ImageNet-21K and the resulted pre-trained ViT-B-16 model on ImageNet-1K has shown a fantastic result (Top-1 84.4%)?

I would love to help with porting the weights if you guys are interested. Is there any recommendation on what information to look for in order to get the job done?

Thank you, John

[ecktoh]

@johnypark I know this is a year late, and might be completely irrelevant now, but if you want to work with ViT pre-trained on ImageNet-21K you can access it through the timm package.

Source: https://github.com/Alibaba-MIIL/ImageNet21K Example: model = timm.create_model('vit_base_patch16_224_miil_in21k', pretrained=True)

Sorry if this wasn't helpful!