kmkurn / pytorch-crf

(Linear-chain) Conditional random field in PyTorch.
https://pytorch-crf.readthedocs.io
MIT License
935 stars 151 forks source link

Add Weights to Sample while calculating the CRF Loss #86

Closed Talisberg closed 2 years ago

Talisberg commented 2 years ago

I have several thousands of sentences with different frequencies in real world on which I'm training a BI-LSTM CRF and using CRF layer.

is there a way to introduce these weights to the CRF layer?

kmkurn commented 2 years ago

I'm afraid not. If you have per-sentence weights, what you may want do is multiply the weights with the output of forward with reduction='none' as argument. If you have per-token weights however, there's no easy way so you need to fork and modify the code yourself.