Keras finetune examples cannot generate hashable key

ZJaume commented 1 year ago

System Info

transformers version: 4.25.1
Platform: Linux-4.15.0-200-generic-x86_64-with-glibc2.17
Python version: 3.8.13
Huggingface_hub version: 0.11.1
PyTorch version (GPU?): 1.10.1+cu102 (True)
Tensorflow version (GPU?): 2.11.0 (True)
Flax version (CPU?/GPU?/TPU?): not installed (NA)
Jax version: not installed
JaxLib version: not installed
Using GPU in script?: Yes
Using distributed or parallel set-up in script?: No

Who can help?

@gante @Rocketknight1

Information

[X] The official example scripts
[ ] My own modified scripts

Tasks

[X] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
[ ] My own task or dataset (give details below)

Reproduction

Steps to reproduce:

Use the Keras example

from transformers import TFAutoModelForSequenceClassification
from transformers import AutoTokenizer
from tensorflow.keras.optimizers import Adam
from datasets import load_dataset
import tensorflow as tf
import numpy as np

dataset = load_dataset("glue", "cola")
dataset = dataset["train"]  # Just take the training split for now

tokenizer = AutoTokenizer.from_pretrained("bert-base-cased")
tokenized_data = tokenizer(dataset["sentence"], return_tensors="np", padding=True)

labels = np.array(dataset["label"])  # Label is already an array of 0 and 1

# Load and compile our model
model = TFAutoModelForSequenceClassification.from_pretrained("bert-base-cased")
# Lower learning rates are often better for fine-tuning transformers
model.compile(optimizer=Adam(3e-5))

model.fit(tokenized_data, labels)

All model checkpoint layers were used when initializing TFBertForSequenceClassification.

Some layers of TFBertForSequenceClassification were not initialized from the model checkpoint at bert-base-cased and are newly initialized: ['classifier']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
No loss specified in compile() - the model's internal loss computation will be used as the loss. Don't panic - this is a common way to train TensorFlow models in Transformers! To disable this behaviour please pass a loss argument, or explicitly pass `loss=None` if you do not want your model to compute a loss.
Traceback (most recent call last):
  File "../test_mirrored.py", line 22, in <module>
    model.fit(tokenized_data, labels)
  File "/home/user/bicleaner-ai-trainings/venv/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 70, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/home/user/bicleaner-ai-trainings/venv/lib/python3.8/site-packages/tensorflow/core/function/polymorphism/function_cache.py", line 117, in lookup
    dispatch_key = self._dispatch_table.dispatch(key)
  File "/home/user/bicleaner-ai-trainings/venv/lib/python3.8/site-packages/tensorflow/core/function/polymorphism/type_dispatch.py", line 78, in dispatch
    if request in self._dispatch_table:
  File "/home/user/bicleaner-ai-trainings/venv/lib/python3.8/site-packages/tensorflow/core/function/polymorphism/function_cache.py", line 77, in __hash__
    return hash((self.call_context, self.function_type))
  File "/home/user/bicleaner-ai-trainings/venv/lib/python3.8/site-packages/tensorflow/core/function/polymorphism/function_type.py", line 246, in __hash__
    return hash(
  File "/home/user/bicleaner-ai-trainings/venv/lib/python3.8/site-packages/tensorflow/core/function/polymorphism/function_type.py", line 106, in __hash__
    return hash((self.name, self.kind, self.optional, self.type_constraint))
  File "/home/user/bicleaner-ai-trainings/venv/lib/python3.8/site-packages/tensorflow/core/function/trace_type/default_types.py", line 207, in __hash__
    return hash(self.components)
  File "/home/user/bicleaner-ai-trainings/venv/lib/python3.8/site-packages/tensorflow/core/function/trace_type/default_types.py", line 207, in __hash__
    return hash(self.components)
  File "/home/user/bicleaner-ai-trainings/venv/lib/python3.8/site-packages/tensorflow/core/function/trace_type/default_types.py", line 584, in __hash__
    return hash((self.identifier, self.base))
ValueError: Cannot generate a hashable key for IteratorSpec(({'input_ids': TensorSpec(shape=(None, 47), dtype=tf.int64, name=None), 'token_type_ids': TensorSpec(shape=(None, 47), dtype=tf.int64, name=None), 'attention_mask': TensorSpec(shape=(None, 47), dtype=tf.int64, name=None)}, TensorSpec(shape=(None,), dtype=tf.int64, name=None)),) because the _serialize() method returned an unsupproted value of type <class 'transformers.tokenization_utils_base.BatchEncoding'>

I also had to change dataset['text'] to dataset['example']

Expected behavior

Train succesfully.

sgugger commented 1 year ago

cc @Rocketknight1

Rocketknight1 commented 1 year ago

Derp, that's a problem with the example, I'll push a fix! The problem is that Keras recognizes dict objects but not our BatchEncoding returned by the tokenizer, even though BatchEncoding is a subclass of dict.

If you replace the last line with model.fit(dict(tokenized_data), labels) it should work.

Rocketknight1 commented 1 year ago

Should be fixed by #20732

baburz commented 1 year ago

@Rocketknight1 I'm getting " 'dict' object is not callable" typeError when I used this solution, any idea why?

Rocketknight1 commented 1 year ago

@baburz Sorry for the delay! Can you paste the exact code you ran?

huggingface / transformers