Open Dorish opened 6 months ago
Hi @Dorish,
Absolutely, you can follow the HF documentation to first download the model to your local machine, and then use the local model path as the model_name
when passing it into LLMLingua.
Hi @Dorish,
Absolutely, you can follow the HF documentation to first download the model to your local machine, and then use the local model path as the
model_name
when passing it into LLMLingua.
Hi @iofu728 iofu728 , is not working when i pass a local model path to LLMLingua, what's wrong?
raise error:
i meet the same problem @iofu728 I think PromptCompressor may not know whether model_name is local or remote
llm_lingua = PromptCompressor(model_name=qwen_path)
ConnectionError: HTTPSConnectionPool(host='openaipublic.blob.core.windows.net', port=443): Max retries exceeded with url: /encodings/cl100k_base.tiktoken (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f07280e31f0>: Failed to establish a new connection: [Errno 101] Network is unreachable'))
i meet the same problem @iofu728 I think PromptCompressor may not know whether model_name is local or remote
llm_lingua = PromptCompressor(model_name=qwen_path)
ConnectionError: HTTPSConnectionPool(host='openaipublic.blob.core.windows.net', port=443): Max retries exceeded with url: /encodings/cl100k_base.tiktoken (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f07280e31f0>: Failed to establish a new connection: [Errno 101] Network is unreachable'))
Have you solved this problem? I want to use LLMLingua-2 locally, so I download the model, but when I set the model_name, it still occur the /encodings/cl100k_base.tiktoken problem
I run it smoothly using local model with absolute path. You'd better check other problems instread of focusing on this project's code.
Describe the issue
Due to slow network, LLMLingua failed to download the model from huggingface during running.
Is it possible that I download the model manually in advance and LLMLinggua load the model from a local location? If it's possible, how to to it?
Thanks in advance!