search

google-research / task_adaptation

Apache License 2.0
311 stars 35 forks source link

How to run VTAB on multi-GPUs? #14

Open woxue opened 3 years ago

woxue commented 3 years ago

Thanks for your solid work! I have some problem with reproducing the result on your leaderboard. The suggest batch size on your paper is 512 however the biggest batch size supported on one GPU(2080 ti) is 64 (128 will cause OOM). The tensorflow_hub seems not working well with tf.distribute.MirroredStrategy() to support multi-GPUs, leading to the error below:

RuntimeError: variable_scope module/ was unused but the corresponding name_scope was already taken. just as mentioned in https://github.com/tensorflow/hub/issues/64

Would you please give me some suggestions? (Not familiar with tf very well)