`AttributeError: 'BertConfig' object has no attribute 'items'` when saving a tf keras model with `transformers 4.12.4`

harupy commented 2 years ago

Environment info

transformers version: 4.12.4
Platform: Linux
Python version: 3.7.10
PyTorch version (GPU?):
Tensorflow version (GPU?): 2.7.0
Using GPU in script?: No
Using distributed or parallel set-up in script?: No

Who can help

Information

Model I am using (Bert, XLNet ...): TFBert

The problem arises when using:

[ ] the official example scripts: (give details below)
[x] my own modified scripts: (give details below)

The tasks I am working on is:

[ ] an official GLUE/SQUaD task: (give the name)
[ ] my own task or dataset: (give details below)

To reproduce

Steps to reproduce the behavior:

Google Colab notebook

Run this code:

import tensorflow as tf
import transformers
import sys

print(sys.version)
print(tf.__version__)
print(transformers.__version__)

bert = transformers.TFBertModel(transformers.BertConfig())
input_ids = tf.keras.layers.Input(shape=(512,), dtype=tf.int32)
model = tf.keras.Model(inputs=[input_ids], outputs=[bert(input_ids).last_hidden_state])
model.compile()
model.save("model")

2021-11-17 19:16:12.651287: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2021-11-17 19:16:12.651307: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
3.7.10
2.7.0
4.12.4
2021-11-17 19:16:13.813364: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory
2021-11-17 19:16:13.813392: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303)
2021-11-17 19:16:13.813402: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (haru-Z590-S01): /proc/driver/nvidia/version does not exist
2021-11-17 19:16:13.813550: I tensorflow/core/platform/cpu_feature_guard.cc:151] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2021-11-17 19:16:20.635357: W tensorflow/python/util/util.cc:368] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
WARNING:absl:Found untraced functions such as embeddings_layer_call_fn, embeddings_layer_call_and_return_conditional_losses, encoder_layer_call_fn, encoder_layer_call_and_return_conditional_losses, pooler_layer_call_fn while saving (showing 5 of 1055). These functions will not be directly callable after loading.
Traceback (most recent call last):
  File "b.py", line 13, in <module>
    model.save("model")
  File "/home/haru/miniconda3/envs/mlflow-dev-env/lib/python3.7/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/home/haru/miniconda3/envs/mlflow-dev-env/lib/python3.7/site-packages/transformers/configuration_utils.py", line 237, in __getattribute__
    return super().__getattribute__(key)
AttributeError: 'BertConfig' object has no attribute 'items'

Expected behavior

harupy commented 2 years ago

In transformers 4.12.3, the attached code works.

harupy commented 2 years ago

After commenting out the changes made in https://github.com/huggingface/transformers/pull/14361, the attached code works in transformers 4.12.4.

https://github.com/huggingface/transformers/blob/b567510cff606c9dd67cb7c56169bc596590e700/src/transformers/modeling_tf_utils.py#L695-L700

cc @sgugger and @LysandreJik who reviewed & approved #14361

Rocketknight1 commented 2 years ago

@harupy The offending PR has been reverted and a patch release has been deployed. We're working on a fix that will resolve the original issues without causing new ones, which will hopefully be deployed soon. In the meantime, your code should work as before!

harupy commented 2 years ago

@Rocketknight1 Got it, thanks!

huggingface / transformers