[詞庫問題回報]

[mjhsieh]

分類

• 詞庫不夠普遍或常用，抑或是需要另外新開專業（愛好）詞彙功能

說明

• ACG 詞: 吉翁... etc.
• nonPG 詞: 汁男... etc.

[tianjianjiang]

也許短期目標是找些奇妙的詞典，像是偽基之類的。

[xatier]

I have this locally to manage multiple user-defined dictionaries, where dict1, dict2, dict3 are coming from different sources/themes.

#!/usr/bin/env bash

set -euo pipefail

OUT="data.txt"

sources=(symbol.txt emoji.txt dict1.txt dict2.txt dict3.txt)

echo '# user phrases file' >"$OUT"
echo "" >>"$OUT"
for f in "${sources[@]}"; do
    cat "$f" >>"$OUT"
    echo "" >>"$OUT"
done

[xatier]

I run jieba.analyse.textrank and jieba.analyse.extract_tags (TF-IDF) to extract phrases from web pages and extend my dictionaries above with some semi-manual scripts.

https://github.com/fxsjy/jieba?tab=readme-ov-file#%E5%9F%BA%E4%BA%8E-tf-idf-%E7%AE%97%E6%B3%95%E7%9A%84%E5%85%B3%E9%94%AE%E8%AF%8D%E6%8A%BD%E5%8F%96