Cannot run local Ollama behind corporate proxy, errors with "502 Bad Gateway" for "openaipublic.blob.core.windows.net"

[csi-lk]

Describe the bug

I assume there's a way to configure no call to be made to the OpenAI servers causing this error but I can't find it

Relevant Continue Config

config = ContinueConfig(
    allow_anonymous_telemetry=False,
    models=Models(
        default=Ollama(
            model="codellama",
            server_url="http://localhost:11434",
            max_context_length=2048,
        ),
        unused=[
            MaybeProxyOpenAI(system_message="", model="gpt-4", api_key=""),
            MaybeProxyOpenAI(system_message="", model="gpt-3.5-turbo", api_key=""),
        ],
    ),
    context_providers=[]
    policy=DefaultPolicy(),
);

To Reproduce

Be on corporate machine that blocks access to openaipublic.blob.core.windows.net

Steps to reproduce the behavior:

1. Install Ollama
2. Install Continue
3. Follow continue docs for setup
4. Put something in textarea and click Continue
5. See error

Console logs

HTTPSConnectionPool(host='openaipublic.blob.core.windows.net', port=443): Max retries exceeded with url: /encodings/cl100k_base.tiktoken (Caused by ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 502 Bad Gateway')))

Additional context

Running the model locally works fine

[sestinj]

This is actually coming from the tiktoken package, which counts tokens, as it downloads the data it needs upon first load. I thought this was a pretty niche issue, but at this point I've seen it enough times to realize something must be done.

Nothing you can do in config file unfortunately to solve this. I'll make a fix today though, so that it can fall back to just counting words if this download fails.

[sestinj]

Also, the pattern seems to be that this issue arises for those using a proxy, is something along these lines true for you? Either workplace firewall, special SSL certs, etc...?

[csi-lk]

Yes, workplace proxy

It seems the url openaipublic.blob.core.windows.net is specifically in the denylist

Thanks for jumping on this so quickly

[sestinj]

Oh wow that's interesting. Curious how long this deny list is for that specifically to be included

I'll let you know once I have an update, hoping not more than a couple hours

[csi-lk]

Sorry, when I say specifically, I mean if I go to the url it denies

They probably have *.core.windows.net or even *windows.net on the denylist

hoping not more than a couple hours

Amazing cheers

[sestinj]

Alright, new version (v0.0.341) is ready, should address this, lmk if not!

[csi-lk]

Just updated, works perfectly

Amazing turnaround, thanks @sestinj