uie-char-small 运行 inference.py 错误

[YYGe01]

您好: 作者您好, 我正在中文数据集上运行 inference.py , 平台为linux, 报错如下: ` Traceback (most recent call last): File "/home/ubuntu/yuyuangen/code/UIE/inference.py", line 163, in main() File "/home/ubuntu/yuyuangen/code/UIE/inference.py", line 94, in main predictor = HuggingfacePredictor( File "/home/ubuntu/yuyuangen/code/UIE/inference.py", line 38, in init self._tokenizer = huggingface_transformers.T5TokenizerFast.from_pretrained( File "/home/ubuntu/anaconda3/envs/uie/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 1719, in from_pretrained return cls._from_pretrained( File "/home/ubuntu/anaconda3/envs/uie/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 1732, in _from_pretrained slow_tokenizer = (cls.slow_tokenizer_class)._from_pretrained( File "/home/ubuntu/anaconda3/envs/uie/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 1791, in _from_pretrained tokenizer = cls(*init_inputs, **init_kwargs) File "/home/ubuntu/anaconda3/envs/uie/lib/python3.8/site-packages/transformers/models/t5/tokenization_t5.py", line 130, in init self.sp_model.Load(vocab_file) File "/home/ubuntu/anaconda3/envs/uie/lib/python3.8/site-packages/sentencepiece/init.py", line 367, in Load return self.LoadFromFile(model_file) File "/home/ubuntu/anaconda3/envs/uie/lib/python3.8/site-packages/sentencepiece/init.py", line 171, in LoadFromFile return _sentencepiece.SentencePieceProcessor_LoadFromFile(self, arg) TypeError: not a string

Process finished with exit code 1 `

配置如下:

import argparse
    parser = argparse.ArgumentParser()
    parser.add_argument(
        '--data', '-d', default='data/text2spotasoc/entity_zh/zh_weibo')
    parser.add_argument(
        '--model', '-m', default='./hf_models/uie-char-small')
    parser.add_argument('--max_source_length', default=256, type=int)
    parser.add_argument('--max_target_length', default=192, type=int)
    parser.add_argument('--batch_size', default=16, type=int)
    parser.add_argument('-c', '--config', dest='map_config',
                        help='Offset Re-mapping Config',
                        default='config/offset_map/closest_offset_zh.yaml')
    parser.add_argument('--decoding', default='spotasoc')
    parser.add_argument('--verbose', action='store_true')
    parser.add_argument('--match_mode', default='normal',
                        choices=['set', 'normal', 'multimatch'])
    options = parser.parse_args()

[YYGe01]

已解决, 替换成自定义的model类即可:

class HuggingfacePredictor:
    def __init__(self, model_path, schema_file, max_source_length=256, max_target_length=192) -> None:
        # self._tokenizer = huggingface_transformers.T5TokenizerFast.from_pretrained(
        #     model_path)
        self._tokenizer = T5BertTokenizer.from_pretrained(model_path)

[YYGe01]

还有个问题是对于中文的unk词, UIE\uie\extraction\predict_parser\utils.py 中的 fix_unk_from_text函数, 好像不兼容.

[luyaojie]

您好，

不兼容是指？