search

Separius / BERT-keras

Keras implementation of BERT with pre-trained weights
GNU General Public License v3.0
813 stars 196 forks source link

BERT on TPU or GPU? #23

Closed ShabRa1365 closed 5 years ago

ShabRa1365 commented 5 years ago

I have a fundamental question about BERT, do I need to run that only on TPU or it is doable for running on GPU and also for CPU as well? Because I am testing the example on my local machine without connecting to my cloud TPU or GPU, and wondering why I face with this error after running tutorial.py: File "/Users/.../BERT-keras/transformer/init.py", line 16, in tpu_compatible from tensorflow.contrib.tpu.python.tpu.keras_support import KerasTPUModel

ModuleNotFoundError: No module named 'tensorflow.contrib' if it will resolve with GPU, I can use eGPU and give it a go.

Separius commented 5 years ago

Hmm, it should be possible to run this on a CPU can you test whether import tensorflow.contrib works on your machine or not?

ShabRa1365 commented 5 years ago

Thanks for the prompt reply. I tried to import tensorflow.contrib , didn't work. It seems that because I am using Tensorflow 2, not be able to use tf.contrib. I even did try run this: import tensorflow.compat.v1 as tf tf.disable_v2_behavior() to convert X1 to X2. Still have ModuleNotFoundError: No module named 'tensorflow.contrib' error.

Separius commented 5 years ago

Can you downgrade? or even better, can you make the code compatible with TF 2? and send a pull request?, it shouldn't be that hard, but I don't know.

ShabRa1365 commented 5 years ago

I was able to run the code on sensor flow 1.13.1 with python version 3.6.5. on GPU. it doesn't necessary required TPU. but it didn't work on the CPU. cheers when I upgrade to version 2 I will send pull.