How can one add weights for different tags?

[kmkurn]

Hi, could you please explain a bit more clearly about this?

[failable]

The document of nn.CrossEntropyLoss says

It is useful when training a classification problem with C classes. If provided, the optional argument weight should be a 1D Tensor assigning weight to each of the classes. This is particularly useful when you have an unbalanced training set.

I wonder if this is possible in the CRF loss layer. Thanks.

[kmkurn]

I see. The CRF does not offer this, but there might be tricks to do this with the current implementation, depending on how you want to weight the tags. For instance, you may incorporate the weights into the emission scores before feeding them into the CRF layer. You need to work out the maths and check if this can achieve the weighting you want.

[allenyummy]

Thanks for @kmkurn 's suggestion.

I have three tags, and I want to weight them with [0.11, 1, 0.16].

If I have a logit [a,b,c] from bert output, did you mean that I can feed [0.11 a, 1 b, 0.16 * c] as emission score into CRF ?

Thanks.

[kmkurn]

Hi @allenyummy. It's been a while since my last post. What you wrote seems to be what I had in mind, but you'll need to verify yourself if it achieves what you want.