Open umoqnier opened 5 years ago
Hello,
I have exactly the same issue, if I am able to train my model with bytes but when I use the tagger if the output is a bytes there is an internal error (same as above) which provide me to get the tag.
The only solution I have for the moment is to use crfsuite instead which is able to output non-ascii tags...
Currently I base my code on this tutorial and I have some problems with
tag
method after the train section. I catch theUnicodeDecodeError
exception like thisThe output looks like this
I tried to decode my
X_test
beforetag
usingdecode('utf-8')
but does seems not to works.Just in case, I had some
UnicodeEncodeError
problems at thetrainer
object as shown below but seems that works usingencode('utf-8')
for every substring. With this method I'm forcing manual encoding before append objects in trainer. This issue is mentioned at #96 and this solution works for me.NOTE: Sorry for my deficent english. I hope I've been clear enough. If not, please tell me!!! :)