Chapter 3: 'BertConfig' object has no attribute 'hidden_state'

[hoangminhtoan]

Information

The question or comment is about chapter:

• [ ] Introduction
• [ ] Text Classification
• [x] Transformer Anatomy
• [ ] Multilingual Named Entity Recognition
• [ ] Text Generation
• [ ] Summarization
• [ ] Question Answering
• [ ] Making Transformers Efficient in Production
• [ ] Dealing with Few to No Labels
• [ ] Training Transformers from Scratch
• [ ] Future Directions

Question or comment

I got this error when running notebook for Chapter03 on google colab with transformers ver 4.13.0

[AlessandroMiola]

Are you sure you're not mixing hidden_size and hidden_state up? hidden_state is just the name which is given to the argument passed to the .forward method in MultiHeadAttention and it is not used as an attribute of the config object as far as I can see.

[hoangminhtoan]

Are you sure you're not mixing hidden_size and hidden_state up? hidden_state is just the name which is given to the argument passed to the .forward method in MultiHeadAttention and it is not used as an attribute of the config object as far as I can see.

The hidden_state`` is called in.forward(self, hidden_state)inAttentionHeadclass. I cloned the notebook and then ran the notebook. I'll change the attributehidden_stateintohidden_size``` to check if the error occurs.

[AlessandroMiola]

I didn't mean to change hidden_state into hidden_size; I meant that as is (and as far as I could experiment) hidden_state is not used as an attribute of the BertConfig object (as BertConfig does not have such an attribute), but rather as an argument to the .forward methods of both AttentionHead and MultiHeadAttention classes, which shouldn't hurt.

This said, I didn't clone and run the notebook directly, therefore I might be wrong.