search

chinhsuanwu / coatnet-pytorch

A PyTorch implementation of "CoAtNet: Marrying Convolution and Attention for All Data Sizes"
https://arxiv.org/abs/2106.04803
MIT License
365 stars 67 forks source link

About the stochastic depth #3

Closed JiaquanYe closed 2 years ago

JiaquanYe commented 2 years ago

Hi, I can't found the code about stochastic depth in your implementation.

And I add the stochastic depth code and train a CoAtNet-Tiny on ImageNet 1k, but got 79.27%@top1.

Have you reproduce the results reported by the paper?

chinhsuanwu commented 2 years ago

Hi @JiaquanYe

This repo is just an implementation of the architecture, and I have not trained on ImageNet. Not sure what you are referring to for "CoAtNet-Tiny", but I guess you did not follow all settings mentioned in the paper, FYI:

img

JiaquanYe commented 2 years ago

Hi @JiaquanYe

This repo is just an implementation of the architecture, and I have not trained on ImageNet. Not sure what you are referring to for "CoAtNet-Tiny", but I guess you did not follow all settings mentioned in the paper, FYI:

img

My mistake... I mean CoAtNet-0. I see stochastic depth in paper and your upload your paper.

OK, I will tune the parameter. Thanks.