search

plkmo / BERT-Relation-Extraction

PyTorch implementation for "Matching the Blanks: Distributional Similarity for Relation Learning" paper
Apache License 2.0
565 stars 132 forks source link

BERT_REL with SciBert #33

Open mariaBio opened 3 years ago

mariaBio commented 3 years ago

HI,

Thank you for your code and for adding BioBert. Tried to use your code with SciBert. For this, I created a code 3 to specify the model type in the input (just as you introduced 2 for BioBert), and added the necessary specification everywhere (following BioBert example).

When training, there is the following error:

12/18/2020 03:02:57 PM [INFO]: Starting training process...

Traceback (most recent call last): File "main_task.py", line 52, in net = train_and_fit(args) File "/mnt/irisgpfs/users/mbiryukov/BertRel-UPD/BERT-Relation-Extraction/src/tasks/trainer.py", line 161, in train_and_fit e1_e2_start=e1_e2_start)
File "/opt/apps/resif/data/production/v1.2-20191021/default/software/devel/PyTorch/1.2.0-fosscuda-2019a-Python-3.7.2/lib/python3.7/site-packages/torch/nn/modules/module.py", line 547, in call result = self.forward(*input, **kwargs) File "/mnt/irisgpfs/users/mbiryukov/BertRel-UPD/BERT-Relation-Extraction/src/model/BERT/modeling_bert.py", line 759, in forward classification_logits = self.classification_layer(v1v2) File "/opt/apps/resif/data/production/v1.2-20191021/default/software/devel/PyTorch/1.2.0-fosscuda-2019a-Python-3.7.2/lib/python3.7/site-packages/torch/nn/modules/module.py", line 591, in getattr type(self).name, name)) AttributeError: 'BertModel' object has no attribute 'classification_layer'

Could you please tell what should be modified (m.b. in model config or train?) to get it work?

Thank you!

vabatista commented 2 years ago

I got the same error trying to use another model other than Bert-base/large-ucased. This occurs because of the constructor of BertModel Class in src/model/BERT/modeling_bert.py.

There's a definition of classification layer only for bert-base|large-ucased.

TharinduDR commented 2 years ago

Did you find a solution for this? I am trying to use the BERT multilingual base, but facing a similar issue.

plkmo commented 9 months ago

Hi all, yes @vabatista is right, the constructor of BertModel has been modified here from the original, in order to implement this model.

If you want to use this for other models that does not use BertModel constructor here, you will have the copy the modification over to the base constructor of other models.