search

Linear95 / bert-intent-slot-detector

BERT-based intent and slots detector for chatbots.
119 stars 15 forks source link

输入包含非中文字符时,槽位输出会包含#号 #2

Closed swxnh closed 7 months ago

swxnh commented 7 months ago

image 如图,当我输入'我想听badapple'时,返回的槽位中包含#字符

Linear95 commented 7 months ago

这个是BERT tokenizer的算法导致的，你可以直接把#过滤掉，可以看下这个解释