Open ctrado18 opened 5 years ago
I have trained a german CRF. I looked at the feature and weights to analyse why and why not the entity is recognised.
E.g. for this sentences. There it recognises correctly the word 'Leistung' but also ''gezahlt" which I don't want.
'wie viel Leistung bekomm ich gezahlt?'
Then I looked at the features predicting the word 'gezahlt':
features [('-1:prefix5', 'ich'), ('-1:suffix3', 'ich'), ('0:bias', 'bias'), ('EOS', True)]
here just suffix3 and prefix5.
The individual weights are (feature, weight, label):
-1:prefix5:ich weight:0.255969 label: U-Leistung -1:prefix5:ich weight:-0.255969 label: O -1:suffix3:ich weight:0.255969 label: U-Leistung -1:suffix3:ich weight:-0.255969 label: O 0:bias:bias weight:1.512877 label: O 0:bias:bias weight:-1.512877 label: U-Leistung
SUM WEIGHT LEISTUNG -1.000939 SUM WEIGHT O 1.000939
This gives in the sum same weights for entitiy Leistung and label O. But why does the word 'gezahlt' has probability of 77% at the end in prediction?
And I thought negative weights are downgrading the proper label (why is then at all this word an entity)
I hope you can clarify a bit?
I have trained a german CRF. I looked at the feature and weights to analyse why and why not the entity is recognised.
E.g. for this sentences. There it recognises correctly the word 'Leistung' but also ''gezahlt" which I don't want.
'wie viel Leistung bekomm ich gezahlt?'
Then I looked at the features predicting the word 'gezahlt':
features [('-1:prefix5', 'ich'), ('-1:suffix3', 'ich'), ('0:bias', 'bias'), ('EOS', True)]
here just suffix3 and prefix5.
The individual weights are (feature, weight, label):
-1:prefix5:ich weight:0.255969 label: U-Leistung -1:prefix5:ich weight:-0.255969 label: O -1:suffix3:ich weight:0.255969 label: U-Leistung -1:suffix3:ich weight:-0.255969 label: O 0:bias:bias weight:1.512877 label: O 0:bias:bias weight:-1.512877 label: U-Leistung
SUM WEIGHT LEISTUNG -1.000939 SUM WEIGHT O 1.000939
This gives in the sum same weights for entitiy Leistung and label O. But why does the word 'gezahlt' has probability of 77% at the end in prediction?
And I thought negative weights are downgrading the proper label (why is then at all this word an entity)
I hope you can clarify a bit?