Closed pinnareet closed 1 month ago
Hi @pinnareet, thanks for raising this issue!
From the linked colab, it looks the error is coming from the bertopic library and the model implementation there. I'd suggest opening an issue in the repo
@amyeroberts, same problem here with only using Transformers (4.41.0)
with torch.no_grad(): outputs = model( input_ids, attention_mask=attention_mask, token_type_ids=None )
I'm using Bert with Torch version 2.2.1
@aymenkrifa Could you share how the model is created and your running env (The running environment: run transformers-cli env
in the terminal and copy-paste the output) ?
Also had this issue when tried to load model using sentence-transformers==2.3.1
with transformers==4.41
. After changing the version transformers==4.37
and setting sentence-transformers==2.7.0
the problem is gone.
Not sure if downgrading transformers
version could help for everyone, but in my case it fixed the issue.
@b5y Thanks for sharing! It seems this might be a compatibility issue then between transformers and sentence transformers cc @tomaarsen
@amyeroberts I don't think there is any compability issue here. The error itself came from transformers
library and it duplicates the error shown in this original post.
UPDATE: it was fine for more than 1 month, just came out recently on Wednesday.
@b5y Could you share a code snippet which reproduces this and full error traceback?
We see the same issue with transformers-4.41.1
:
0:00:00.410061 INFO: Traceback (most recent call last):
File "alphadia/cli.py", line 333, in run
plan = Plan(
^^^^^
File "alphadia/planning.py", line 126, in __init__
self.load_library()
File "alphadia/planning.py", line 229, in load_library
spectral_library = pept_deep_prediction(spectral_library)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "alphadia/libtransform.py", line 40, in __call__
return self.forward(*args)
^^^^^^^^^^^^^^^^^^^
File "alphadia/libtransform.py", line 306, in forward
model_mgr = ModelManager(device=device)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "peptdeep/pretrained_models.py", line 300, in __init__
self.ms2_model:pDeepModel = pDeepModel(mask_modloss=mask_modloss, device=device)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "peptdeep/model/ms2.py", line 385, in __init__
self.build(
File "peptdeep/model/model_interface.py", line 248, in build
self.model = model_class(**kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "peptdeep/model/ms2.py", line 171, in __init__
self.hidden_nn = building_block.Hidden_HFace_Transformer(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "peptdeep/model/building_block.py", line 224, in __init__
self.bert = BertEncoder(self.config)
^^^^^^^^^^^^^^^^^^^^^^^^
File "transformers/models/bert/modeling_bert.py", line 643, in __init__
self.layer = nn.ModuleList([BertLayer(config) for _ in range(config.num_hidden_layers)])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "transformers/models/bert/modeling_bert.py", line 643, in <listcomp>
self.layer = nn.ModuleList([BertLayer(config) for _ in range(config.num_hidden_layers)])
^^^^^^^^^^^^^^^^^
File "transformers/models/bert/modeling_bert.py", line 558, in __init__
self.attention = BertAttention(config)
^^^^^^^^^^^^^^^^^^^^^
File "transformers/models/bert/modeling_bert.py", line 476, in __init__
self.self = BERT_SELF_ATTENTION_CLASSES[config._attn_implementation](
^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: '_Pseudo_Bert_Config' object has no attribute '_attn_implementation'
0:00:00.410134 ERROR: '_Pseudo_Bert_Config' object has no attribute '_attn_implementation'
@GeorgWa Could you share a reproducible code snippet? From the traceback, it looks like a non-transformers config file is being used (we don't have a _Pseudo_Bert_Config
class)
Yes, will do
Sorry for the confusion, it looks like adding _attn_implementation
to the config object solves the issue on our side.
I'm still unsure why we have to add a private config field, Will check with my colleauge as I'm not an expert in this part of the codebase.
I would expect _Pseudo_Bert_Config
should be inherit the default config or maybe a check if the provided config is an instance.
Thank you!
from transformers.models.bert.modeling_bert import BertEncoder
class _Pseudo_Bert_Config:
def __init__(self,
hidden_dim=256,
intermediate_size=1024,
num_attention_heads=8,
num_bert_layers=4,
dropout=0.1,
output_attentions=False,
):
self.add_cross_attention = False
self.chunk_size_feed_forward = 0
self.is_decoder = False
self.seq_len_dim = 1
self.training = False
self.hidden_act = "gelu"
self.hidden_dropout_prob = dropout
self.attention_probs_dropout_prob = dropout
self.hidden_size = hidden_dim
self.initializer_range = 0.02
self.intermediate_size = intermediate_size
self.layer_norm_eps = 1e-8
self.num_attention_heads = num_attention_heads
self.num_hidden_layers = num_bert_layers
self.output_attentions = output_attentions
#self._attn_implementation = "eager" uncommenting solves the issue
bert = BertEncoder(_Pseudo_Bert_Config())
@GeorgWa It's assumed the config passed to the transformers model is a child of PretrainedConfig. When this config is created (either directly or through AutoModel calls) then the private attribute _attn_implementation
will be set
I got the same error when working in Databricks for model serving, the code works well locally, but when create a endpoint, I got this error:
"error_code": "BAD_REQUEST", "message": "Encountered an unexpected error while evaluating the model. Verify that the input is compatible with the model for inference. Error ''BertModel' object has no attribute 'attn_implementation''"
The same code I deployed a few weeks ago, it worked fine. I tried to downgrade transformers version, it did not work.
any tips? Thanks!
Hi @liqi6811, could you a reproducible code snippet? In particular, one which can run locally but fails when deploying on databricks?
@amyeroberts I worked it out by specifying transformers and sentence-transformers version when log model.
# Start an MLflow run and log the model
with mlflow.start_run():
mlflow.pyfunc.log_model("model",
python_model=wrapped_model,
input_example=df_sample,
signature=signature,
pip_requirements=["transformers==4.37",
"sentence-transformers==2.7.0",
"mlflow==2.5.0"])
@liqi6811 Thanks for clarifying cc @tomaarsen re sentence-transformers <-> transformers compatibility
Experiencing the same issue with transformers==4.40.2 and sentence_transformers==3.0.1.
Anyone figure out a workaround?
EDIT: Looks like upgrading to transformers==4.41.2 solved the issue for me.
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.
Please note that issues that do not follow the contributing guidelines are likely to be ignored.
I solved on my end when I downgrade transformers from4.42.3 --> 4.40.2
my other related libraries version
sentence-transformers 3.0.1
transformer-engine 1.8.0+37280ec
System Info
Google Colaboratory
transformers
version: 4.41.0Who can help?
No response
Information
Tasks
examples
folder (such as GLUE/SQuAD, ...)Reproduction
https://colab.research.google.com/drive/1ncTM6lk6ZOfnwSFr_jaXL2zwwvWA-JRj?usp=sharing
Expected behavior
It is not supposed to have an error.