[Question]: access to public storage https://openaipublic.blob.core.windows.net/ is prohibited in secure environments ,

microsoft / LLMLingua

To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.

https://llmlingua.com/

MIT License

4.27k stars 228 forks source link

[Question]: access to public storage https://openaipublic.blob.core.windows.net/ is prohibited in secure environments , #106

Open amrosalehms opened 4 months ago

amrosalehms commented 4 months ago

Describe the issue

In certain environments where private connectivity prohibits access to public storage at https://openaipublic.blob.core.windows.net/, we encountered a connection timeout error when attempting to import the library. Is there a workaround to avoid accessing this public storage account?

iofu728 commented 4 months ago

Hi @amrosalehms, I suspect the issue is due to loading the tokenizer for GPT-3.5-turbo. You can comment out this line and related code to resolve the issue.

Dorish commented 4 months ago

Hi @amrosalehms, I suspect the issue is due to loading the tokenizer for GPT-3.5-turbo. You can comment out this line and related code to resolve the issue.

Hi @iofu728 ,

I'v met the same issue. Is it ok to just comment out the lines you suggested? Will it cause any side effect?

iofu728 commented 3 months ago

Hi @Dorish, the code invoking self.oai_tokenizer, such as in #712, needs to be modified.