TypeError: Cannot convert [array([322., 1.])] to EagerTensor of dtype int64

[pavi-ninjaac]

System Info

transformers.version # 4.5.1 !python -V # Python 3.8.10 Operating System: Ubuntu 20.04.6 LTS

I have faced the following issue:

File ~/projects/Natural-Language-Processing/venv_text_summarizaition/lib/python3.8/site-packages/tensorflow/python/framework/constant_op.py:275, in _constant_impl(value, dtype, shape, name, verify_shape, allow_broadcast)
    273     with trace.Trace("tf.constant"):
    274       return _constant_eager_impl(ctx, value, dtype, shape, verify_shape)
--> 275   return _constant_eager_impl(ctx, value, dtype, shape, verify_shape)
    277 const_tensor = ops._create_graph_constant(  # pylint: disable=protected-access
    278     value, dtype, shape, name, verify_shape, allow_broadcast
    279 )
    280 return const_tensor

File ~/projects/Natural-Language-Processing/venv_text_summarizaition/lib/python3.8/site-packages/tensorflow/python/framework/constant_op.py:285, in _constant_eager_impl(ctx, value, dtype, shape, verify_shape)
    283 def _constant_eager_impl(ctx, value, dtype, shape, verify_shape):
    284   """Creates a constant on the current device."""
--> 285   t = convert_to_eager_tensor(value, ctx, dtype)
    286   if shape is None:
    287     return t

File ~/projects/Natural-Language-Processing/venv_text_summarizaition/lib/python3.8/site-packages/tensorflow/python/framework/constant_op.py:98, in convert_to_eager_tensor(value, ctx, dtype)
     96     dtype = dtypes.as_dtype(dtype).as_datatype_enum
     97 ctx.ensure_initialized()
---> 98 return ops.EagerTensor(value, ctx.device_name, dtype)

TypeError: Cannot convert [array([322.,   1.])] to EagerTensor of dtype int64
<tf.Tensor: shape=(2, 2), dtype=int64, numpy=
array([[322,   1],
       [322,   1]])>

kind of figured it out and made a PR for it : RP:31109 please help me through this and check the PR

Who can help?

No response

Information

• [ ] The official example scripts
• [ ] My own modified scripts

Tasks

• [ ] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• [ ] My own task or dataset (give details below)

Reproduction

from transformers import AutoTokenizer, TFAutoModelForSeq2SeqLM

model_checkpoint = "google/mt5-small"

model_checkpoint = "google/pegasus-cnn_dailymail"

tokenizer = AutoTokenizer.from_pretrained(model_checkpoint) model = TFAutoModelForSeq2SeqLM.from_pretrained(model_checkpoint, from_pt=True) from transformers import DataCollatorForSeq2Seq

data_collator = DataCollatorForSeq2Seq(tokenizer, model=model, return_tensors="tf") features = [tokenized_data["train"][i] for i in range(2)] data_collator(features) tf_train_dataset = model.prepare_tf_dataset( tokenized_data["train"], collate_fn=data_collator, shuffle=True, batch_size=8, )

Expected behavior

I would expect it to give a dataset which could be used with tensorflow.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[amyeroberts]

Closing as #31109 was merged in