search

THUDM / WebGLM

WebGLM: An Efficient Web-enhanced Question Answering System (KDD 2023)
Apache License 2.0
1.57k stars 135 forks source link

运行报错 TypeError: expected str, bytes or os.PathLike object, not NoneType #67

Open YangWulve opened 1 year ago

YangWulve commented 1 year ago

WebGLM Initializing... Traceback (most recent call last): File "/home/e222/YWT/WebGLM-main/web_demo.py", line 63, in webglm = load_model(args) File "/home/e222/YWT/WebGLM-main/model/modeling_webglm.py", line 66, in load_model webglm = WebGLM(webglm_ckpt_path, retiever_ckpt_path, args.device, args.filter_max_batch_size, args.searcher) File "/home/e222/YWT/WebGLM-main/model/modeling_webglm.py", line 9, in init self.tokenizer = AutoTokenizer.from_pretrained(webglm_ckpt_path, trust_remote_code=True) File "/home/e222/anaconda3/envs/webglm/lib/python3.9/site-packages/transformers/models/auto/tokenization_auto.py", line 693, in from_pretrained return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, *kwargs) File "/home/e222/anaconda3/envs/webglm/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1812, in from_pretrained return cls._from_pretrained( File "/home/e222/anaconda3/envs/webglm/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1975, in _from_pretrained tokenizer = cls(init_inputs, **init_kwargs) File "/home/e222/anaconda3/envs/webglm/lib/python3.9/site-packages/transformers/models/gpt2/tokenization_gpt2.py", line 188, in init with open(vocab_file, encoding="utf-8") as vocab_handle: TypeError: expected str, bytes or os.Path Like object, not NoneType

疑似webglm_ckpt_path = args.webglm_ckpt_path or os.getenv("WEBGLM_CKPT") or 'THUDM/WebGLM'没有找到，应该修改哪里呢？

hanyullai commented 1 year ago

您好，看起来问题应该是checkpoint的路径设定有问题，能告诉我们WEBGLM_CKPT的环境变量值吗？

YangWulve commented 1 year ago

您好，看起来问题应该是checkpoint的路径设定有问题，能告诉我们WEBGLM_CKPT的环境变量值吗？

你好，按照readme步骤，我没有设置WEBGLM_CKPT，不知道是否有huggingface默认地址。

hanyullai commented 1 year ago

有可能是本地环境变量无意设置了WEBGLM_CKPT导致的错误。可以尝试手动设置下WEBGLM_CKPT环境变量或传入参数webglm_ckpt_path。详细可见：https://github.com/THUDM/WebGLM/blob/74d3bcac3b1b7d4ec35e21a33677f5ed92a08fed/model/modeling_webglm.py#L56

YangWulve commented 1 year ago

请问有官方仓库吗？不知道这里手动设置路径该选择哪里，感谢。 p.s. 本地变量等检测无误

hanyullai commented 1 year ago

有的，我们的官方仓库是https://huggingface.co/THUDM/WebGLM 和 https://huggingface.co/THUDM/WebGLM-2B 。也可以从这里进行下载。

YangWulve commented 1 year ago

不好意思，修改为官方仓库后还是同样的报错。并且本地的WEBGLM_CKPT路径为None，默认应该从huggingface上下载。请问还有可能是哪些地方的问题呢？ 命令行： python cli_demo.py -w THUDM/WebGLM --searcher bing