search

lsdefine / attention-is-all-you-need-keras

A Keras+TensorFlow Implementation of the Transformer: Attention Is All You Need
708 stars 188 forks source link

startup error #35

Open rafaleo opened 4 years ago

rafaleo commented 4 years ago

Hello. I try to evaluate your script and got the following error message:

(base) C:\Users\cp\Python\attention-is-all-you-need-keras>python en2de_main.py
Using TensorFlow backend.
loading data/en2de_word.txt
loading data/en2de.h5
loading data/en2de.valid.h5
seq 1 words: 3369
seq 2 words: 3665
train shapes: (29000, 43) (29000, 47)
valid shapes: (1014, 34) (1014, 39)
2020-03-11 13:08:06.384900: I tensorflow/core/common_runtime/process_util.cc:147] Creating new thread pool with default
inter op setting: 2. Tune using inter_op_parallelism_threads for best performance.
Traceback (most recent call last):
  File "en2de_main.py", line 33, in <module>
    s2s.compile(Adam(0.001, 0.9, 0.98, epsilon=1e-9))
  File "C:\Users\cp\Python\attention-is-all-you-need-keras\transformer.py", line 452, in compile
    loss = get_loss(final_output, tgt_true)
  File "C:\Users\cp\Python\attention-is-all-you-need-keras\transformer.py", line 440, in get_loss
    loss = tf.nn.sparse_softmax_cross_entropy_with_logits(labels=y_true, logits=y_pred)
  File "C:\Users\cp\Anaconda3\lib\site-packages\tensorflow_core\python\ops\nn_ops.py", line 3537, in sparse_softmax_cros
s_entropy_with_logits_v2
    labels=labels, logits=logits, name=name)
  File "C:\Users\cp\Anaconda3\lib\site-packages\tensorflow_core\python\ops\nn_ops.py", line 3470, in sparse_softmax_cros
s_entropy_with_logits
    array_ops.shape(logits)[:-1]))
  File "C:\Users\cp\Anaconda3\lib\site-packages\tensorflow_core\python\ops\check_ops.py", line 658, in assert_equal
    data, summarize, message, name)
  File "C:\Users\cp\Anaconda3\lib\site-packages\tensorflow_core\python\ops\check_ops.py", line 333, in _binary_assert
    if condition:
  File "C:\Users\cp\Anaconda3\lib\site-packages\tensorflow_core\python\framework\ops.py", line 757, in __bool__
    self._disallow_bool_casting()
  File "C:\Users\cp\Anaconda3\lib\site-packages\tensorflow_core\python\framework\ops.py", line 526, in _disallow_bool_ca
sting
    self._disallow_in_graph_mode("using a `tf.Tensor` as a Python `bool`")
  File "C:\Users\cp\Anaconda3\lib\site-packages\tensorflow_core\python\framework\ops.py", line 515, in _disallow_in_grap
h_mode
    " this function with @tf.function.".format(task))
tensorflow.python.framework.errors_impl.OperatorNotAllowedInGraphError: using a `tf.Tensor` as a Python `bool` is not al
lowed in Graph execution. Use Eager execution or decorate this function with @tf.function.

I use keras 2+ and tf 2+ as well but not using any gpu.

banayoyo commented 4 years ago

do u have solve it? I meet the same error

salihgunduz commented 4 years ago

no. not yet.

30 Nis 2020 Per 05:49 tarihinde banayoyo notifications@github.com şunu yazdı:

do u have solve it? I meet the same error

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/Lsdefine/attention-is-all-you-need-keras/issues/35#issuecomment-621581123, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJCO3UTUC6DCCK6JFDUCP3TRPDRNBANCNFSM4LFUNUVQ .

WangLi2019Gt commented 3 years ago

do u have solve it? I meet the same error

xux1984-china commented 3 years ago

do you solve the error,I have the same problem

xux1984-china commented 3 years ago

E:\ProgramData\Anaconda3\envs\Dlearning\python.exe F:/reaserch/Dlearning/attention-is-all-you-need-keras-master/pinyin_main.py 2021-11-02 11:35:20.786849: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cudart64_101.dll loading data/pinyin_word.txt seq 1 words: 198 seq 2 words: 218 2021-11-02 11:35:22.790685: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library nvcuda.dll 2021-11-02 11:35:22.822934: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties: pciBusID: 0000:01:00.0 name: GeForce RTX 2070 Super computeCapability: 7.5 coreClock: 1.38GHz coreCount: 40 deviceMemorySize: 8.00GiB deviceMemoryBandwidth: 417.29GiB/s 2021-11-02 11:35:22.823133: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cudart64_101.dll 2021-11-02 11:35:22.826677: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cublas64_10.dll 2021-11-02 11:35:22.829716: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cufft64_10.dll 2021-11-02 11:35:22.831096: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library curand64_10.dll 2021-11-02 11:35:22.834603: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cusolver64_10.dll 2021-11-02 11:35:22.836756: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cusparse64_10.dll 2021-11-02 11:35:22.843372: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cudnn64_7.dll 2021-11-02 11:35:22.843524: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0 2021-11-02 11:35:22.843780: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN)to use the following CPU instructions in performance-critical operations: AVX2 To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2021-11-02 11:35:22.850462: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x296289b0a10 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2021-11-02 11:35:22.850703: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2021-11-02 11:35:22.850868: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties: pciBusID: 0000:01:00.0 name: GeForce RTX 2070 Super computeCapability: 7.5 coreClock: 1.38GHz coreCount: 40 deviceMemorySize: 8.00GiB deviceMemoryBandwidth: 417.29GiB/s 2021-11-02 11:35:22.851063: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cudart64_101.dll 2021-11-02 11:35:22.851157: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cublas64_10.dll 2021-11-02 11:35:22.851249: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cufft64_10.dll 2021-11-02 11:35:22.851343: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library curand64_10.dll 2021-11-02 11:35:22.851436: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cusolver64_10.dll 2021-11-02 11:35:22.851529: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cusparse64_10.dll 2021-11-02 11:35:22.851621: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cudnn64_7.dll 2021-11-02 11:35:22.851731: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0 2021-11-02 11:35:23.321532: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1257] Device interconnect StreamExecutor with strength 1 edge matrix: 2021-11-02 11:35:23.321709: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1263] 0 2021-11-02 11:35:23.321789: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1276] 0: N 2021-11-02 11:35:23.321964: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1402] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6613 MB memory) -> physical GPU (device: 0, name: GeForce RTX 2070 Super, pci bus id: 0000:01:00.0, compute capability: 7.5) 2021-11-02 11:35:23.324097: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x2964d88f920 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2021-11-02 11:35:23.324220: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): GeForce RTX 2070 Super, Compute Capability 7.5 Traceback (most recent call last): File "F:/reaserch/Dlearning/attention-is-all-you-need-keras-master/pinyin_main.py", line 23, in s2s.compile(opt) File "F:\reaserch\Dlearning\attention-is-all-you-need-keras-master\transformer.py", line 453, in compile loss = get_loss(final_output, tgt_true) File "F:\reaserch\Dlearning\attention-is-all-you-need-keras-master\transformer.py", line 441, in get_loss loss = tf.nn.sparse_softmax_cross_entropy_with_logits(labels=y_true, logits=y_pred) File "E:\ProgramData\Anaconda3\envs\Dlearning\lib\site-packages\tensorflow\python\util\dispatch.py", line 201, in wrapper return target(*args, kwargs) File "E:\ProgramData\Anaconda3\envs\Dlearning\lib\site-packages\tensorflow\python\ops\nn_ops.py", line 4176, in sparse_softmax_cross_entropy_with_logits_v2 labels=labels, logits=logits, name=name) File "E:\ProgramData\Anaconda3\envs\Dlearning\lib\site-packages\tensorflow\python\util\dispatch.py", line 201, in wrapper return target(*args, *kwargs) File "E:\ProgramData\Anaconda3\envs\Dlearning\lib\site-packages\tensorflow\python\ops\nn_ops.py", line 4108, in sparse_softmax_cross_entropy_with_logits array_ops.shape(logits)[:-1])) File "E:\ProgramData\Anaconda3\envs\Dlearning\lib\site-packages\tensorflow\python\util\dispatch.py", line 201, in wrapper return target(args, kwargs) File "E:\ProgramData\Anaconda3\envs\Dlearning\lib\site-packages\tensorflow\python\ops\check_ops.py", line 671, in assert_equal data, summarize, message, name) File "E:\ProgramData\Anaconda3\envs\Dlearning\lib\site-packages\tensorflow\python\ops\check_ops.py", line 335, in _binary_assert if condition: File "E:\ProgramData\Anaconda3\envs\Dlearning\lib\site-packages\tensorflow\python\framework\ops.py", line 877, in bool self._disallow_bool_casting() File "E:\ProgramData\Anaconda3\envs\Dlearning\lib\site-packages\tensorflow\python\framework\ops.py", line 490, in _disallow_bool_casting self._disallow_in_graph_mode("using a tf.Tensor as a Python bool") File "E:\ProgramData\Anaconda3\envs\Dlearning\lib\site-packages\tensorflow\python\framework\ops.py", line 479, in _disallow_in_graph_mode " this function with @tf.function.".format(task)) tensorflow.python.framework.errors_impl.OperatorNotAllowedInGraphError: using a tf.Tensor as a Python bool is not allowed in Graph execution. Use Eager execution or decorate this function with @tf.function.

yqw123 commented 2 years ago

I have the same error

lsdefine commented 2 years ago

I can run the code in tf=2.6.0. Please provide the environments.

Or using a lambda layer to package the tf function may help, like this: transformer.py 453-457

loss = get_loss(final_output, tgt_true)

    loss = Lambda(lambda x:get_loss(x[0], x[1]))([final_output, tgt_true])
    self.ppl = K.exp(loss)
    #self.accu = get_accu(final_output, tgt_true)
    self.accu = Lambda(lambda x:get_accu(x[0], x[1]))([final_output, tgt_true])

This is also runable in my env.

thibault-roux commented 2 years ago

I had the same problem but correct it by upgrading tensorflow to 2.7.0 :)