search

openai / tiktoken

tiktoken is a fast BPE tokeniser for use with OpenAI's models.
MIT License
11.03k stars 748 forks source link

SSLError: HTTPSConnectionPool(host='openaipublic.blob.core.windows.net', port=443): Max retries exceeded with url #281

Open sijiashen opened 2 months ago

sijiashen commented 2 months ago

Issue I ran encoding = tiktoken.get_encoding("cl100k_base") and encountered the following error: SSLError: HTTPSConnectionPool(host='openaipublic.blob.core.windows.net', port=443): Max retries exceeded with url: /encodings/cl100k_base.tiktoken (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1129)')))

Cause & Solution Our hypothesis is that we are not allowed to making a request to a host outside of the kubeflow VPC. Q1: Would you say this is the primary cause?

If the answer to Q1 is yes: Q2: Does tiktoken allow us to read from a file system artifact instead of making a web call?

polinabee commented 2 months ago

I have the same issue, did you find any resolution?

anithamudigoudar commented 2 months ago

I have the same issue, is there any update on this?

polinabee commented 2 months ago

I seem to be able to get rid of this issue by setting a more restrictive SSL token in openssl dir for my python

anithamudigoudar commented 2 months ago

@polinabee Can you please provide the steps to do it please

songwang41 commented 1 month ago

I also encountered this error? any solution?

nkilm commented 1 month ago

It's because of the network issue. You can manually download the .tiktoken file and specify the location of this file in tiktoken using TIKTOKEN_CACHE_DIR environment variable.

You can find a detailed guide here - https://stackoverflow.com/questions/76106366/how-to-use-tiktoken-in-offline-mode-computer