allenai / sequential_sentence_classification

https://arxiv.org/pdf/1909.04054
Apache License 2.0
77 stars 27 forks source link

Prediction with CRF for true test data #12

Open ItikaGupta opened 3 years ago

ItikaGupta commented 3 years ago

Hi,

Thank you for sharing the code for your work. I have been able to run the code and train the models for all the three cases. When I try to run predictor.py, it works fine for BERT and (BERT + Transformer) models. However, when using predictor.py for a (BERT + Transformer + CRF) model, since we don't have true labels for test data, the below code gives error:

  ` if self.with_crf:

        mask_sentences = (labels != -1)

        best_paths = self.crf.viterbi_tags(label_logits, mask_sentences)`

I understand why it gives an error, but was not sure how or what to modify in order for it to work.

Any suggestion will be really helpful.

Thank you!

ItikaGupta commented 3 years ago

I solved it by using sentence's bert tensors to create the label masks.