how to implement cosub training use deit-III

[xiaoguang-1]

Hi, i want to use cosub trainging method for custom images classification work, but i don't find implemention in the code. Does this work contains the details of cosub training method. If yes, my computer can't support RASamper ,does it have an influence? Best, xiaoguang

[xiaoguang-1]

Trouble you again, I would like to know how the residual function at efficient stochastic depth is implemented? Thank you so much! yours best, xiaoguang

[TouvronHugo]

Hi @xiaoguang-1 , Thanks for your message. I have added cosub in the code you need to use stochastic depth with cosub but RASampler is not mandatory. I still need to add the efficient version of Stochastic depth but the code works without it. Best, Hugo