search

fxsjy / jieba

结巴中文分词
MIT License
33.41k stars 6.73k forks source link

feat: add parameter logTF in extract_tags to determine whether to weight TF #1023

Open RC-Kanashii opened 3 months ago

RC-Kanashii commented 3 months ago

Sometimes the increase in a word's TF is not linearly correlated with the importance of the word. Therefore, weighting TF with log2 may have a better effect.