Add Swin-Transformer to keras.applications

[innat]

If you open a GitHub issue, here is our policy:

It must be a bug, a feature request, or a significant problem with the documentation (for small docs fixes please send a PR instead). The form below must be filled out.

Here's why we have that policy:.

Keras developers respond to issues. We want to focus on work that benefits the whole community, e.g., fixing bugs and adding features. Support only helps individuals. GitHub also notifies thousands of people when issues are filed. We want them to see you communicating an interesting problem, rather than being redirected to Stack Overflow.

System information.

TensorFlow version (you are using): 2.6 Are you willing to contribute it (Yes/No): Yes

Describe the feature and the current behavior/state.

Describe the feature clearly here. Be sure to convey here why the requested feature is needed. Any brief description of the use-case would help.

Paper: Swin Transformer: Hierarchical Vision Transformer using Shifted Windows Original Code: https://github.com/microsoft/Swin-Transformer?utm_source=catalyzex.com

It's a variant of the transformer model and achieves state-of-the-art performance or comparable performance with the best CNN-based models. It also contains enough citations (~250 at this moment) for addition to the package.

On ImageNet-1K and 22K, below is the comparable results with EfficientNet (CNN) models.

-	Img Size	Top 1K acc	-	Img Size	Top 1K acc	Top 22K acc
E3	300	81.6	EfficientNetV2-S	-	83.9	84.9
E5	456	83.6	EfficientNetV2-M	-	85.1	86.2
E7	600	84.3	EfficientNetV2-L	-	85.7	86.8
-	-	-	EfficientNetV2-XL	-	-	87.3
Swin-T	224	81.3	Swin-B	224	-	85.2
Swin-S	224	83.0	Swin-B	384	-	86.4
Swin-B	224	83.5	Swin-L	384	-	87.3
Swin-B	384	84.5	-	-	-	-

Will this change the current api? How? Yes. It will change as follows

tensorflow.keras.applications.SwinT
tensorflow.keras.applications.SwinS
tensorflow.keras.applications.SwinB
tensorflow.keras.applications.SwinL

Who will benefit from this feature? Keras users.

Contributing

• Do you want to contribute a PR? (yes/no): yes.
• If yes, please read this page for instructions
• Briefly describe your candidate solution(if contributing):

[qlzh727]

Thanks for this FR, could you provide the link to the original paper so that we can take a look?

In general, I think we are open accept new application models if they are widely recognized. Currently we might host it in keras.application, but will very likely to move them to keras-cv repository, which is targeted for CV specific components.

[qlzh727]

Just FYI, you can take a look for the new guidance for adding application to keras in https://github.com/keras-team/keras/pull/15447.

[innat]

@qlzh727 The paper link is provided.

I'm really excited about the keras-cv repo. Any good news?

[qlzh727]

So far we are scoping it and draft the roadmap for it. Will have more details later this year.

[bhack]

@Rishit-dagli are you interested to contribute here?

Context: https://github.com/tensorflow/models/issues/10269

[qlzh727]

Thanks for the reference. If you would like to contribute, you are welcome to send it to keras CV, which will be new home for cv related application model.

[Rishit-dagli]

@Rishit-dagli are you interested to contribute here?

Context: tensorflow/models#10269

Thanks @bhack , I would definitely love to take this up!

So far we are scoping it and draft the roadmap for it. Will have more details later this year.

@qlzh727 Please do let me know if this should be contributed to keras.applications or keras-cv .

Until then I will start reading the contributing guidelines for both and take a look at a couple of past PRs as well.

[innat]

@qlzh727 any update on this. luke informed us that the new model will be placed to keras-application for now. FYI, there is already a version 2 of swin-transformer.

[qlzh727]

Sorry for the late reply. We probably have to host this in the keras/application for now. Eventually we will move this to keras-cv, but we are not ready yet. Feel free to send PR if you have any.

[innat]

@qlzh727 Thanks for clarifying. cc. @Rishit-dagli

[innat]

@qlzh727 Could you please move this issue to keras-cv? I think it's more fit there now.

[qlzh727]

Thanks for the notice.

[tanzhenyu]

It seems I created a duplicate issue. Yes we'd like to accept the contribution as a keras-application style model. Is there any way to split the work, for example, porting on SWIN-T Base first?

[bhack]

Can we close this for https://github.com/keras-team/keras-cv/issues/671?