Open shin04 opened 3 years ago
it seems to secure pseudo large batch size
https://akichan-f.medium.com/%E5%A4%A7%E3%81%8D%E3%81%AA%E3%83%9F%E3%83%8B%E3%83%90%E3%83%83%E3%83%81%E3%81%AE%E5%8A%B9%E5%8A%9B%E3%81%A8-tensorflow%E3%82%92%E4%BD%BF%E3%81%A3%E3%81%A6%E5%A4%A7%E3%81%8D%E3%81%AA%E3%83%9F%E3%83%8B%E3%83%90%E3%83%83%E3%83%81%E3%82%92%E5%AD%A6%E7%BF%92%E3%81%95%E3%81%9B%E3%82%8B%E6%96%B9%E6%B3%95-gradient-accumulation-3147d89eb43f
it seems to secure pseudo large batch size
seems to be helpful
https://akichan-f.medium.com/%E5%A4%A7%E3%81%8D%E3%81%AA%E3%83%9F%E3%83%8B%E3%83%90%E3%83%83%E3%83%81%E3%81%AE%E5%8A%B9%E5%8A%9B%E3%81%A8-tensorflow%E3%82%92%E4%BD%BF%E3%81%A3%E3%81%A6%E5%A4%A7%E3%81%8D%E3%81%AA%E3%83%9F%E3%83%8B%E3%83%90%E3%83%83%E3%83%81%E3%82%92%E5%AD%A6%E7%BF%92%E3%81%95%E3%81%9B%E3%82%8B%E6%96%B9%E6%B3%95-gradient-accumulation-3147d89eb43f