search

shibing624 / pycorrector

pycorrector is a toolkit for text error correction. 文本纠错，实现了Kenlm，T5，MacBERT，ChatGLM3，Qwen2.5等模型应用在纠错场景，开箱即用。
https://www.mulanai.com/product/corrector/
Apache License 2.0
5.57k stars 1.1k forks source link

Macbert怎么导入本地模型？ #478

Closed yuanjingtai closed 8 months ago

yuanjingtai commented 8 months ago

Describe the Question

Please provide a clear and concise description of what the question is. 我的网络环境有问题，请问怎么导入已经下载好的本地模型？

shibing624 commented 8 months ago

写入model path就行，参考https://github.com/shibing624/pycorrector/blob/master/pycorrector/macbert/macbert_corrector.py#L25 指定为绝对路径。

yuanjingtai commented 8 months ago

thx

bucaiLi commented 7 months ago

描述问题

请简明扼要地说明问题是什么。 我的网络环境有问题，请问怎么导入已经下载好的本地模型？

你好 请问你解决这个问题了嘛 我是那怕到了本地路径，还是从网上下载，老哥你是怎么解决的呀

137315198 commented 3 months ago

Describe the Question

Please provide a clear and concise description of what the question is. 我的网络环境有问题，请问怎么导入已经下载好的本地模型？

你好，请问你的模型权重是在哪里下载的？

shibing624 commented 3 months ago

hf models

137315198 commented 3 months ago

hf models

下载指定为绝对路径后，加载时会提示以下错误 Traceback (most recent call last): File "/Users/zyb/Downloads/pycorrector-master/pycorrector/macbert/demo2.py", line 36, in main() File "/Users/zyb/Downloads/pycorrector-master/pycorrector/macbert/demo2.py", line 14, in main m = MacBertCorrector() File "/Users/zyb/Downloads/pycorrector-master/pycorrector/macbert/macbert_corrector.py", line 28, in init self.model = BertForMaskedLM.from_pretrained(model_name_or_path) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/transformers/modeling_utils.py", line 3613, in from_pretrained with safe_open(resolved_archive_file, framework="pt") as f: safetensors_rust.SafetensorError: Error while deserializing header: HeaderTooLarge

137315198 commented 3 months ago

hf models

下载指定为绝对路径后，加载时会提示以下错误 Traceback (most recent call last): File "/Users/zyb/Downloads/pycorrector-master/pycorrector/macbert/demo2.py", line 36, in main() File "/Users/zyb/Downloads/pycorrector-master/pycorrector/macbert/demo2.py", line 14, in main m = MacBertCorrector() File "/Users/zyb/Downloads/pycorrector-master/pycorrector/macbert/macbert_corrector.py", line 28, in init self.model = BertForMaskedLM.from_pretrained(model_name_or_path) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/transformers/modeling_utils.py", line 3613, in from_pretrained with safe_open(resolved_archive_file, framework="pt") as f: safetensors_rust.SafetensorError: Error while deserializing header: HeaderTooLarge

已解决