Closed SanJoseCosta closed 7 months ago
Epoch 1/30
Traceback (most recent call last):
File "transformer_train.py", line 408, in
Detected at node 'gradient_tape/transformer/transformer_encoder/multi_head_attention/softmax/add/BroadcastGradientArgs' defined at (most recent call last):
File "transformer_train.py", line 408, in
I have the same problem as SanJoseCosta. It works in Google Colab (Tensorflow version 2.9.2) but not on my own computer running tensorflow 2.11.0.
2022-12-08 05:53:18.266030: W tensorflow/core/kernels/data/cache_dataset_ops.cc:856] The calling iterator did not fully read the dataset being cached. In order to avoid unexpected truncation of the dataset, the partially cached contents of the dataset will be discarded. This can happen if you have an input pipeline similar to dataset.cache().take(k).repeat()
. You should use dataset.take(k).cache().repeat()
instead.
@fchollet Would you please help?
I just installed tensorflow version 2.9.2 and it works.
I still get this warning, so it must not be the problem:
2022-12-08 06:51:10.808714: W tensorflow/core/kernels/data/cache_dataset_ops.cc:856] The calling iterator did not fully read the dataset being cached. In order to avoid unexpected truncation of the dataset, the partially cached contents of the dataset will be discarded. This can happen if you have an input pipeline similar to dataset.cache().take(k).repeat()
. You should use dataset.take(k).cache().repeat()
instead.
With 2.11.0 I get this error just before the above warning:
File "/home/ben/miniconda3/envs/tf/lib/python3.10/site-packages/keras/engine/base_layer.py", line 1132, in __call__
outputs = call_fn(inputs, *args, **kwargs)
File "/home/ben/miniconda3/envs/tf/lib/python3.10/site-packages/keras/utils/traceback_utils.py", line 96, in error_handler
return fn(*args, **kwargs)
File "/home/ben/miniconda3/envs/tf/lib/python3.10/site-packages/keras/layers/activation/softmax.py", line 95, in call
inputs += adder
Node: 'transformer/transformer_encoder/multi_head_attention/softmax/add' 2 root error(s) found. (0) INVALID_ARGUMENT: required broadcastable shapes [[{{node transformer/transformer_encoder/multi_head_attention/softmax/add}}]] [[broadcast_weights_1/assert_broadcastable/is_valid_shape/else/_1/broadcast_weights_1/assert_broadcastable/is_valid_shape/has_valid_nonscalar_shape/then/_53/broadcast_weights_1/assert_broadcastable/is_valid_shape/has_valid_nonscalar_shape/has_invalid_dims/concat/_94]] (1) INVALID_ARGUMENT: required broadcastable shapes [[{{node transformer/transformer_encoder/multi_head_attention/softmax/add}}]] 0 successful operations. 0 derived errors ignored. [Op:__inference_train_function_17025]
I don't want to be stuck on an old version of Tensorflow for ever more though, so it would be great if someone would help me get it working even though I have a temporary solution.
Hi,
The example has been updated to Keras 3 and it is working fine without any issue.
Here is the link to the updated tutorial: https://keras.io/examples/nlp/neural_machine_translation_with_transformer/
This issue is stale because it has been open for 14 days with no activity. It will be closed if no further activity occurs. Thank you.
This issue was closed because it has been inactive for 28 days. Please reopen if you'd like to work on this further.
python3 put/es.py