Closed danyaljj closed 2 years ago
Update: based on the following warning:
I0321 19:57:29.230169 28674 tpu_initializer_helper.cc:94] libtpu.so already in use by another process. Run "$ sudo lsof -w /dev/accel0" to figure out which process is using the TPU. Not attempting to load libtpu.so in this process.
WARNING:absl:No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)
jax devices: 1
it seems that some other processes (probably my earlier runs) are using TPUs. So I killed the process shown in sudo lsof -w /dev/accel0
.
These are the shards that I downloaded from here: https://mystic.the-eye.eu/public/AI/GPT-J-6B/previous_checkpoints/step_384500/