search

martinsbruveris / tensorflow-image-models

TensorFlow port of PyTorch Image Models (timm) - image models with pretrained weights.
https://tfimm.readthedocs.io/en/latest/
Apache License 2.0
286 stars 25 forks source link

Just want to ask a question for education purposes... hope it's okay! #59

Closed slerman12 closed 2 years ago

slerman12 commented 2 years ago

Hi, this library is great. I'm wondering, maybe you can tell me this with your experience, what would be your small-list of the "top" essentials of a state-of-art image model these days? Data augmentation of course (mixup?), regularization (layer norm?), EMA, weight decay... I want to get a minimalist competitive ImageNet model working.

martinsbruveris commented 2 years ago

A good question. The honest answer is: I don't know. I have not trained an ImageNet model myself. I have trained a bunch of other models for specific applications, but not ImageNet. Having said that, I would look at the ConvNeXt paper, in particular Table 5. They are quite explicit when it comes to describing the training regime.

And then there is the ResNet strikes back paper, which is also very detailed on how train your ResNet to be competitive on ImageNet. I hope that helps.

slerman12 commented 2 years ago

Thanks. I'm building a library and was wondering if you had input.

slerman12 commented 2 years ago

Please let me know if you think of any additions that would be absolute essentials - I have loosely read both of those papers, but I'll take a deeper look.

Implementation is hard, even when you know the recipe 😆

martinsbruveris commented 2 years ago

Implementation is always hard... Otherwise coding wouldn't be so much fun. For a certain notion of fun...

Good luck with your library! Is it open source?

slerman12 commented 2 years ago

TBA, haha