AttributeError: 'BertModel' object has no attribute 'attn_implementation'

pinnareet commented 3 months ago

System Info

Google Colaboratory

transformers version: 4.41.0
Platform: Linux-6.1.85+-x86_64-with-glibc2.35
Python version: 3.10.12
Huggingface_hub version: 0.23.0
Safetensors version: 0.4.3
Accelerate version: not installed
Accelerate config: not found
PyTorch version (GPU?): 2.3.0+cu121 (True)
Tensorflow version (GPU?): 2.15.0 (True)
Flax version (CPU?/GPU?/TPU?): 0.8.3 (gpu)
Jax version: 0.4.26
JaxLib version: 0.4.26
Using GPU in script?: Yes
Using distributed or parallel set-up in script?: No

Who can help?

No response

Information

[ ] The official example scripts
[X] My own modified scripts

Tasks

[ ] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
[X] My own task or dataset (give details below)

Reproduction

https://colab.research.google.com/drive/1ncTM6lk6ZOfnwSFr_jaXL2zwwvWA-JRj?usp=sharing

Expected behavior

It is not supposed to have an error.

amyeroberts commented 3 months ago

Hi @pinnareet, thanks for raising this issue!

From the linked colab, it looks the error is coming from the bertopic library and the model implementation there. I'd suggest opening an issue in the repo

aymenkrifa commented 3 months ago

@amyeroberts, same problem here with only using Transformers (4.41.0)

with torch.no_grad(): outputs = model( input_ids, attention_mask=attention_mask, token_type_ids=None )

I'm using Bert with Torch version 2.2.1

amyeroberts commented 3 months ago

@aymenkrifa Could you share how the model is created and your running env (The running environment: run transformers-cli env in the terminal and copy-paste the output) ?

b5y commented 3 months ago

Also had this issue when tried to load model using sentence-transformers==2.3.1 with transformers==4.41. After changing the version transformers==4.37 and setting sentence-transformers==2.7.0 the problem is gone.

Not sure if downgrading transformers version could help for everyone, but in my case it fixed the issue.

amyeroberts commented 3 months ago

@b5y Thanks for sharing! It seems this might be a compatibility issue then between transformers and sentence transformers cc @tomaarsen

b5y commented 3 months ago

@amyeroberts I don't think there is any compability issue here. The error itself came from transformers library and it duplicates the error shown in this original post.

UPDATE: it was fine for more than 1 month, just came out recently on Wednesday.

amyeroberts commented 3 months ago

@b5y Could you share a code snippet which reproduces this and full error traceback?

GeorgWa commented 3 months ago

We see the same issue with transformers-4.41.1:

0:00:00.410061 INFO: Traceback (most recent call last):
  File "alphadia/cli.py", line 333, in run
    plan = Plan(
           ^^^^^
  File "alphadia/planning.py", line 126, in __init__
    self.load_library()
  File "alphadia/planning.py", line 229, in load_library
    spectral_library = pept_deep_prediction(spectral_library)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "alphadia/libtransform.py", line 40, in __call__
    return self.forward(*args)
           ^^^^^^^^^^^^^^^^^^^
  File "alphadia/libtransform.py", line 306, in forward
    model_mgr = ModelManager(device=device)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "peptdeep/pretrained_models.py", line 300, in __init__
    self.ms2_model:pDeepModel = pDeepModel(mask_modloss=mask_modloss, device=device)
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "peptdeep/model/ms2.py", line 385, in __init__
    self.build(
  File "peptdeep/model/model_interface.py", line 248, in build
    self.model = model_class(**kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^
  File "peptdeep/model/ms2.py", line 171, in __init__
    self.hidden_nn = building_block.Hidden_HFace_Transformer(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "peptdeep/model/building_block.py", line 224, in __init__
    self.bert = BertEncoder(self.config)
                ^^^^^^^^^^^^^^^^^^^^^^^^
  File "transformers/models/bert/modeling_bert.py", line 643, in __init__
    self.layer = nn.ModuleList([BertLayer(config) for _ in range(config.num_hidden_layers)])
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "transformers/models/bert/modeling_bert.py", line 643, in <listcomp>
    self.layer = nn.ModuleList([BertLayer(config) for _ in range(config.num_hidden_layers)])
                                ^^^^^^^^^^^^^^^^^
  File "transformers/models/bert/modeling_bert.py", line 558, in __init__
    self.attention = BertAttention(config)
                     ^^^^^^^^^^^^^^^^^^^^^
  File "transformers/models/bert/modeling_bert.py", line 476, in __init__
    self.self = BERT_SELF_ATTENTION_CLASSES[config._attn_implementation](
                                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: '_Pseudo_Bert_Config' object has no attribute '_attn_implementation'

0:00:00.410134 ERROR: '_Pseudo_Bert_Config' object has no attribute '_attn_implementation'

amyeroberts commented 3 months ago

@GeorgWa Could you share a reproducible code snippet? From the traceback, it looks like a non-transformers config file is being used (we don't have a _Pseudo_Bert_Config class)

GeorgWa commented 3 months ago

Yes, will do

GeorgWa commented 3 months ago

Sorry for the confusion, it looks like adding _attn_implementation to the config object solves the issue on our side. I'm still unsure why we have to add a private config field, Will check with my colleauge as I'm not an expert in this part of the codebase.

I would expect _Pseudo_Bert_Config should be inherit the default config or maybe a check if the provided config is an instance.

Thank you!

from transformers.models.bert.modeling_bert import BertEncoder

class _Pseudo_Bert_Config:
    def __init__(self, 
        hidden_dim=256, 
        intermediate_size=1024,
        num_attention_heads=8,
        num_bert_layers=4,
        dropout=0.1,
        output_attentions=False,
    ):
        self.add_cross_attention = False
        self.chunk_size_feed_forward = 0
        self.is_decoder = False
        self.seq_len_dim = 1
        self.training = False
        self.hidden_act = "gelu"
        self.hidden_dropout_prob = dropout
        self.attention_probs_dropout_prob = dropout
        self.hidden_size = hidden_dim
        self.initializer_range = 0.02
        self.intermediate_size = intermediate_size
        self.layer_norm_eps = 1e-8
        self.num_attention_heads = num_attention_heads
        self.num_hidden_layers = num_bert_layers
        self.output_attentions = output_attentions
        #self._attn_implementation = "eager" uncommenting solves the issue

bert = BertEncoder(_Pseudo_Bert_Config())

amyeroberts commented 3 months ago

@GeorgWa It's assumed the config passed to the transformers model is a child of PretrainedConfig. When this config is created (either directly or through AutoModel calls) then the private attribute _attn_implementation will be set

liqi6811 commented 3 months ago

I got the same error when working in Databricks for model serving, the code works well locally, but when create a endpoint, I got this error:

"error_code": "BAD_REQUEST", "message": "Encountered an unexpected error while evaluating the model. Verify that the input is compatible with the model for inference. Error ''BertModel' object has no attribute 'attn_implementation''"

The same code I deployed a few weeks ago, it worked fine. I tried to downgrade transformers version, it did not work.

any tips? Thanks!

amyeroberts commented 3 months ago

Hi @liqi6811, could you a reproducible code snippet? In particular, one which can run locally but fails when deploying on databricks?

liqi6811 commented 3 months ago

@amyeroberts I worked it out by specifying transformers and sentence-transformers version when log model.

# Start an MLflow run and log the model
with mlflow.start_run():
    mlflow.pyfunc.log_model("model", 
                            python_model=wrapped_model, 
                            input_example=df_sample, 
                            signature=signature,
                            pip_requirements=["transformers==4.37", 
                                              "sentence-transformers==2.7.0", 
                                              "mlflow==2.5.0"])

amyeroberts commented 3 months ago

@liqi6811 Thanks for clarifying cc @tomaarsen re sentence-transformers <-> transformers compatibility

Wmuntean commented 2 months ago

Experiencing the same issue with transformers==4.40.2 and sentence_transformers==3.0.1.

Anyone figure out a workaround?

EDIT: Looks like upgrading to transformers==4.41.2 solved the issue for me.

github-actions[bot] commented 1 month ago

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

ECMGit commented 1 month ago

I solved on my end when I downgrade transformers from4.42.3 --> 4.40.2 my other related libraries version

sentence-transformers     3.0.1
transformer-engine        1.8.0+37280ec

huggingface / transformers