search

chroma-core / chroma

the AI-native open-source embedding database
https://www.trychroma.com/
Apache License 2.0
14.6k stars 1.22k forks source link

[Bug]: HttpClient connection is very slow from Windows client to Linux server #2795

Open anderatt opened 5 days ago

anderatt commented 5 days ago

What happened?

I expect the connection between the application and the Chromadb server to occur in every thousandth of a second, but in my case it is around 1:10 (mm:ss)

Versions

Client Side

Server Side

Relevant log output

I don't get an error during the connection process, only the connection time is extremely high. 
However, I tested connectivity from the Linux client to the Linux server and it was extremely fast (< 120ms)

192.168.0.5 is my server running a chromadb server

Start Connection time:-  2024-09-13 14:53:30.196609
Finish Connection time:- 2024-09-13 14:54:37.814719

=====  Python Code  =================================

import chromadb
import datetime

print("Start Connection time:-", datetime.datetime.now())

chroma_client = chromadb.HttpClient(host='192.168.0.5', port=8000)

print("Finish Connection time:-", datetime.datetime.now())
jeffchuber commented 3 days ago

@anderatt can you try to curl to /api/v1/heartbeat/ on your server from your windows machine and see if that is fast? that will help us reproduce. thanks!

tazarov commented 2 days ago

@anderatt, adding on top of what @jeffchuber mentioned, also add -v flag so that you can see headers and timestamps in the curl output:

culr -v http://192.168.0.5:8000/api/v1/heartbeat

If you don't have curl available either thru WLS or minGW you can use powershell command as follows:

Invoke-WebRequest -Uri "http://localhost:8000/api/v1/heartbeat" -Verbose

There are many reasons why requests might be slow, granted some of them can be from Chroma client or server.

Some background information about how the Chroma client works. When you create a new client chromadb.HttpClient(...), the client will attempt to connect to Chroma server immediately by verifying that tenant and database exist (that's two separate requests to the server):

INFO:     [16-09-2024 07:50:48] Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
INFO:     [16-09-2024 07:50:51] 192.168.65.1:30408 - "GET /api/v1/tenants/default_tenant HTTP/1.1" 200
INFO:     [16-09-2024 07:50:51] 192.168.65.1:30408 - "GET /api/v1/databases/default_database?tenant=default_tenant HTTP/1.1" 200

Let me ask a few clarifying questions:

anderatt commented 2 days ago

@jeffchuber and @tazarov

Thanks for answer me.

So, let provide more information for you:

Answer for the command : curl -v http://192.168.0.5: 8000 Invoke-WebRequest : Cannot find a positional parameter that accepts the argument '8000'. Row:1 caractere:1

PS C:\Users\ander> curl -v http://192.168.0.5:8000 DETAILED MODE: GET with 0-byte payload curl : {"detail":"Not Found"} No linha:1 caractere:1

The next command works fine!! The execution were in miliseconds PS C:\Users\ander> _Invoke-WebRequest -Uri "http://192.168.0.5:8000/api/v1/heartbeat" -Verbose_ MODO DETALHADO: GET with 0-byte payload MODO DETALHADO: received 44-byte response of content type application/json

StatusCode : 200 StatusDescription : OK Content : {"nanosecond heartbeat":1726492632483973753} RawContent : HTTP/1.1 200 OK Content-Length: 44 Content-Type: application/json Date: Mon, 16 Sep 2024 13:17:09 GMT Server: uvicorn

                {"nanosecond heartbeat":1726492632483973753}

Forms : {} Headers : {[Content-Length, 44], [Content-Type, application/json], [Date, Mon, 16 Sep 2024 13:17:09 GMT], [Server, uvicorn]} Images : {} InputFields : {} Links : {} ParsedHtml : mshtml.HTMLDocumentClass RawContentLength : 44

How is your machine connected to the server: home, corporate etc. network?

I'm connecting in two ways, the first is the one I've used most, it's installing a VM on Oracle VirtualBox with Ubuntu 24.04 installed. The second, once the problems appeared, was to create the entire VM configuration on a LAPTOP on the same network, even so the problems persist and are identical

Is your machine in a corporate network? - frequently in such networks there are proxies and additional firewalls

It is not a corporate network and there are no firewalls or proxies between the machines or between the Windows machine and the VM. The Oracle VirtualBox is installed on the same development Windows machine.

Does your computer have any configured proxies?

On the Windows machine there is the default firewall and Antivirus that comes with the Windows installation, but I tested with both disabled and the problem persists.

Do you have any firewall rules or an anti-virus program that might be inspecting the traffic?

As mentioned previously, even with the standard Windows firewall and antivirus disabled, the problem persists. I do not have any type of firewall or additional antivirus on the development machine

The command to start the Chromadb in a server side is:

/bin/bash source /home/tomzitz/.venv/bin/activate /home/tomzitz/.venv/bin/chroma run --host 0.0.0.0 --port 8000 --path /home/tomzitz/chromadata

tazarov commented 2 days ago

@anderatt Can you test with this:

python -c "import httpx; print(httpx.get('http://192.168.0.5:8000/api/v1/heartbeat').text)"

Make sure you have your virtual environment sourced.

You can also add timestamp printouts. Chroma recently switched from requests to httpx lib so I wonder if it can be something related to that.

anderatt commented 2 days ago

@tazarov

This is the result test:

(venv) PS C:> python -c "import httpx; print(httpx.get('http://192.168.0.5:8000/api/v1/heartbeat').text)" {"nanosecond heartbeat":1726498916083741537}

The elapsed time was 28.120 seconds (28 seconds and 120 milliseconds)