CoAtNet not present in `timm`

[sayakpaul]

@gustheman @sayannath

It turns out that timm doesn't have any implementation of CoAtNet which means we really don't have any pre-trained weights available for the model at the moment. It was an oversight on my end to confuse this model with CoAtNet.

We can do either of the following now:

• Continue with the CoAtNet implementation and train it on smaller datasets to confirm it works. I mention smaller datasets because that way we can complete things within the timeline which doesn't factor in ImageNet-1k related tasks at the moment.
• Decide on another image classification model and follow the same plan as CoAtNet.

Let's discuss.

[sayannath]

Cool! On it.

[sayakpaul]

What are you on? Didn't get you.

[sayannath]

On implementing the model asap.

[sayakpaul]

No, I think it's better to wait for Gus's comments here. Since the pre-trained model parameters aren't available we need to take an informed decision.

[gustheman]

I'm fine with changing to another model to make this achievable! Any suggestions of other possible ones not on TF Hub yet?

[sayannath]

MobileViT can be an option.

[sayakpaul]

Yes sounds good to me. MobileViTs are available in official capacity as well.

@gustheman WDYT?

[gustheman]

I agree, let's go with MobileViT then!