baichuan-inc / Baichuan-7B

A large-scale 7B pretraining language model developed by BaiChuan-Inc.
https://huggingface.co/baichuan-inc/baichuan-7B
Apache License 2.0
5.67k stars 506 forks source link

[Question] 是否有对Tasks.word_segmentation 任务的分词示例代码 #108

Open luxiaobai007 opened 1 year ago

luxiaobai007 commented 1 year ago

Required prerequisites

Questions

def word_segment():
    word_segment_zh = pipeline(task=Tasks.word_segmentation, model='baichuan-inc/baichuan-7B', model_revision='v1.0.2')
    result = word_segment_zh("结婚的和尚未结婚的")
    print(result)

可以提供一下关于分词的示例代码吗

Checklist