下载 tokenizer 时不能指定 auth token

[SakigamiYang]

models/embedding.py 和 models/reranker.py 的init里面都有如下代码：

self.tokenizer = AutoTokenizer.from_pretrained(model_name_or_path) # 这里没有 kwargs self.model = AutoModel.from_pretrained(model_name_or_path, kwargs)

由于下载tokenizer文件时没有传**kwargs，所以指定的 use_auth_token 没有传递进去，下载报错。

Cannot access gated repo for url https://huggingface.co/maidalun1020/bce-embedding-base_v1/resolve/main/tokenizer_config.json. Access to model maidalun1020/bce-embedding-base_v1 is restricted. You must be authenticated to access it.

[shenlei1020]

感谢指出～

[shenlei1020]

已更新