search

snapthat / TF-T5-text-to-text

This repository demonstrate training T5 transformers using tensorflow 2
MIT License
12 stars 2 forks source link

ValueError: None values not supported. in Transformers 4.1.1 #2

Open rk0033 opened 3 years ago

rk0033 commented 3 years ago

@HarrisDePerceptron

I ran the following source on Transformers 4.1.1 and got an error "ValueError: None values not supported.". How should I solve it? https://github.com/snapthat/TF-T5-text-to-text/blob/master/snapthatT5/notebooks/TF-T5-Datasets%20Training.ipynb

Environment info

Information

I get an error when I run as following.

model.fit(tf_train_ds, epochs=5, steps_per_epoch=steps, callbacks=callbacks, 
          validation_data=tf_valid_ds, validation_steps=valid_steps, initial_epoch=epochs_done)

The error message is as following.

Epoch 1/5
WARNING:tensorflow:AutoGraph could not transform <bound method Socket.send of <zmq.sugar.socket.Socket object at 0x7f96900c8660>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: <cyfunction Socket.send at 0x7f96a78fbe58> is not a module, class, method, function, traceback, frame, or code object
To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert
WARNING: AutoGraph could not transform <bound method Socket.send of <zmq.sugar.socket.Socket object at 0x7f96900c8660>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: <cyfunction Socket.send at 0x7f96a78fbe58> is not a module, class, method, function, traceback, frame, or code object
To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert
The parameters `output_attentions`, `output_hidden_states` and `use_cache` cannot be updated when calling a model.They have to be set to True/False in the config object (i.e.: `config=XConfig.from_pretrained('name', output_attentions=True)`).
WARNING:tensorflow:AutoGraph could not transform <function wrap at 0x7f96a52a48c8> and will run it as-is.
Cause: while/else statement not yet supported
To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert
The parameter `return_dict` cannot be set in graph mode and will always be set to `True`.
WARNING: AutoGraph could not transform <function wrap at 0x7f96a52a48c8> and will run it as-is.
Cause: while/else statement not yet supported
To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert
The parameters `output_attentions`, `output_hidden_states` and `use_cache` cannot be updated when calling a model.They have to be set to True/False in the config object (i.e.: `config=XConfig.from_pretrained('name', output_attentions=True)`).
The parameter `return_dict` cannot be set in graph mode and will always be set to `True`.
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-25-bde07b6bc2d5> in <module>()
      1 epochs_done = 0
      2 model.fit(tf_train_ds, epochs=5, steps_per_epoch=steps, callbacks=callbacks, 
----> 3           validation_data=tf_valid_ds, validation_steps=valid_steps, initial_epoch=epochs_done)

9 frames
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/func_graph.py in wrapper(*args, **kwargs)
    975           except Exception as e:  # pylint:disable=broad-except
    976             if hasattr(e, "ag_error_metadata"):
--> 977               raise e.ag_error_metadata.to_exception(e)
    978             else:
    979               raise

ValueError: in user code:

    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:805 train_function  *
        return step_function(self, iterator)
    <ipython-input-6-cee9d76f5377>:12 train_step  *
        outputs = self(x, training=True)
    /usr/local/lib/python3.6/dist-packages/transformers/models/t5/modeling_tf_t5.py:1309 call  *
        decoder_outputs = self.decoder(
    /usr/local/lib/python3.6/dist-packages/transformers/models/t5/modeling_tf_t5.py:733 call  *
        layer_outputs = layer_module(
    /usr/local/lib/python3.6/dist-packages/transformers/models/t5/modeling_tf_t5.py:515 call  *
        cross_attention_outputs = self.layer[1](
    /usr/local/lib/python3.6/dist-packages/transformers/models/t5/modeling_tf_t5.py:424 call  *
        attention_output = self.EncDecAttention(
    /usr/local/lib/python3.6/dist-packages/transformers/models/t5/modeling_tf_t5.py:329 call  *
        position_bias = tf.zeros((1, self.n_heads, real_seq_length, key_length), dtype=tf.float32)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/util/dispatch.py:201 wrapper  **
        return target(*args, **kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/array_ops.py:2819 wrapped
        tensor = fun(*args, **kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/array_ops.py:2877 zeros
        shape = ops.convert_to_tensor(shape, dtype=dtypes.int32)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/profiler/trace.py:163 wrapped
        return func(*args, **kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py:1540 convert_to_tensor
        ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/array_ops.py:1525 _autopacking_conversion_function
        return _autopacking_helper(v, dtype, name or "packed")
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/array_ops.py:1460 _autopacking_helper
        constant_op.constant(elem, dtype=dtype, name=str(i)))
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/constant_op.py:265 constant
        allow_broadcast=True)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/constant_op.py:283 _constant_impl
        allow_broadcast=allow_broadcast))
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/tensor_util.py:445 make_tensor_proto
        raise ValueError("None values not supported.")

    ValueError: None values not supported.
TimoMatzen commented 3 years ago

Is there already a fix for this? I am having the same issue :(

rk0033 commented 3 years ago

No, I haven't been able to solve this problem yet ...