Need Help, getting errors when running Bert-CRF on English words

[lkqnaruto]

Hi

I'm trying to use crf layer together with Bert, but I'm keeping getting the following errors which really confused me. I printed "tags" out, and see the first element in "tags" is -100. Obviously this is the reason that causes the error, but how to fix it?

    numerator = self._compute_score(emissions, tags, mask)
  File "./torchcrf/__init__.py", line 186, in _compute_score
    score = self.start_transitions[tags[0]]
IndexError: index -100 is out of bounds for dimension 0 with size 3
  0%|          | 0/5574 [00:07<?, ?it/s]

[kmkurn]

Hi, the fix is to not use -100 for your padding tags. Just set it to 0. If you pass the correct mask tensor, it will compute things correctly. Padding index of -100 is only relevant if you use cross-entropy loss from PyTorch. It's only meant for convenience.

[venti07]

Hi, I have the same error message. However, if I add 0, then I do have a problem with the labels, since 0 stands for another label. How can I work around the problem, so I can use the CRF layer. Can you be more specific about what I need to change? Thanks in advance!

[kmkurn]

@venti07 You need to keep track which positions hold the dummy label -100 before setting them to 0. After you compute e.g., the best sequence with the CRF layer, you can restore them back using that information. That's what the mask tensor is also useful for.