Separius / BERT-keras

Keras implementation of BERT with pre-trained weights
GNU General Public License v3.0
813 stars 196 forks source link

ValueError: You need tensorflow >= 1.3 for better keras tpu support! #13

Closed astariul closed 5 years ago

astariul commented 5 years ago

When running the example notebook (on Colab), I receive the following error :

ValueError: You need tensorflow >= 1.3 for better keras tpu support!

This error appear when I execute :

g_bert.compile('adam', 'mse')

Any idea from where it comes from ?


I checked the version of Tensorflow with :

!pip list | grep tensorflow

And it gives me an expected result :

mesh-tensorflow 0.0.5
tensorflow 1.12.0
tensorflow-hub 0.2.0
tensorflow-metadata 0.9.0
tensorflow-probability 0.5.0

@HighCWu

HighCWu commented 5 years ago

In fact, the implementation of some keras operations on the TPU has not been completed in tf 1.2, but implemented in tf 1.3. I remember I have added

pip install tf-nightly

at the colab first line. It's used to install tensorflow unstable version 1.3 @Colanim

HighCWu commented 5 years ago

This seems to be my fault, I forgot to correct the previous misuse, now the colab pip install code has been fixed @Colanim

astariul commented 5 years ago

Thanks you @HighCWu for your quick fix !

Now I have another problem : When executing this line with TPU :

from transformer import refresh_keras_backend

The runtime die, I don't really know why.

With GPU, everything seems to work fine, but runtime will die at :

g_bert.train_on_batch(x,y)

CPU works fine.

Do you have similar results ? Maybe I didn't configure my notebook correctly.


EDIT

Everything works fine now. I am not sure of what I changed however..

HighCWu commented 5 years ago

Glad you solved this problem so you can close this issue. If you have some new problems, you can open a new issue @Colanim