search

chroma-core / chroma

the AI-native open-source embedding database
https://www.trychroma.com/
Apache License 2.0
14.42k stars 1.2k forks source link

[Install issue]: How to initialize the tenant and database when using a kubernetes cluster? #2502

Open leobenkel opened 1 month ago

leobenkel commented 1 month ago

What happened?

So I am using chromadb docker image:

ARG VECTOR_DB_ENGINE_VERSION
FROM chromadb/chroma:${VECTOR_DB_ENGINE_VERSION}

ENV IS_PERSISTENT=true
ENV PERSIST_DIRECTORY=/chroma/chroma
ENV ANONYMIZED_TELEMETRY=false
ENV CHROMA_SERVER_NOFILE=4096

ENV CHROMA_SERVER_AUTH_CREDENTIALS_PROVIDER="chromadb.auth.token.TokenConfigServerAuthCredentialsProvider"
ENV CHROMA_SERVER_AUTH_PROVIDER="chromadb.auth.token.TokenAuthServerProvider"
ENV CHROMA_CLIENT_AUTH_PROVIDER="chromadb.auth.token_authn.TokenAuthClientProvider"

ARG VECTOR_DB_HEADER_TOKEN_TRANSPORT
ENV CHROMA_SERVER_AUTH_TOKEN_TRANSPORT_HEADER=VECTOR_DB_HEADER_TOKEN_TRANSPORT
ENV CHROMA_AUTH_TOKEN_TRANSPORT_HEADER=VECTOR_DB_HEADER_TOKEN_TRANSPORT

ARG VECTOR_DB_HEADER_AUTH_CREDS
ENV CHROMA_SERVER_AUTH_CREDENTIALS=VECTOR_DB_HEADER_AUTH_CREDS
ENV CHROMA_CLIENT_AUTH_CREDENTIALS=VECTOR_DB_HEADER_AUTH_CREDS

EXPOSE 8000

COPY chroma_log_config.yml /chroma/chromadb/log_config.yml

and i am hitting it from the other image.

the connection is working but then i am getting:

ValueError: Could not connect to tenant <tenant>. Are you sure it exists?

When I did the same process for posgresql, I just had to add few ENV variable in the db dockerfile to match the back-end ones. I have been searching for few hours and I dont see how to do the same thing for chromadb. How do i setup the tenant / database names to match between the two services ?

Thank you !

Also, does ANONYMIZED_TELEMETRY=false turn off telemetry ? or keep on just not anonymized ? I want telemetry completely off. Also, it seems the auth token is not working as expected. i am able to hit chroma without token and get something back. Also, it seems CHROMA_SERVER_NOFILE is not doing anything. I am still getting WARNING: [11-07-2024 10:57:13] chroma_server_nofile is set to 65535, but this is less than current soft limit of 1048576. chroma_server_nofile will not be set. in the log.

Versions

docker image: 0.5.4

Relevant log output

No response

tazarov commented 1 month ago

@leobenkel, can you look up the error stack to see the actual error that you're getting from Chroma?

leobenkel commented 1 month ago

thank you for your response @tazarov !

here is the full log from the chroma docker:

Traceback (most recent call last):
  File "/chroma/chromadb/server/fastapi/__init__.py", line 76, in catch_exceptions_middleware
    return await call_next(request)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
    raise app_exc
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
    await self.app(scope, receive_or_disconnect, send_no_error)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
    with collapse_excgroups():
  File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
    raise exc
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
    response = await self.dispatch_func(request, call_next)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/chroma/chromadb/server/fastapi/__init__.py", line 90, in check_http_version_middleware
    return await call_next(request)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
    raise app_exc
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
    await self.app(scope, receive_or_disconnect, send_no_error)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 65, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 756, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 776, in app
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 297, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 77, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 72, in app
    response = await func(request)
               ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 278, in app
    raw_response = await run_endpoint_function(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
    return await dependant.call(**values)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/chroma/chromadb/telemetry/opentelemetry/__init__.py", line 133, in async_wrapper
    return await f(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/chroma/chromadb/server/fastapi/__init__.py", line 480, in get_tenant
    await to_thread.run_sync(
  File "/usr/local/lib/python3.11/site-packages/anyio/to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 2177, in run_sync_in_worker_thread
    return await future
           ^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 859, in run
    result = context.run(func, *args)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/chroma/chromadb/telemetry/opentelemetry/__init__.py", line 146, in wrapper
    return f(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^
  File "/chroma/chromadb/api/segment.py", line 140, in get_tenant
    return self._sysdb.get_tenant(name=name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/chroma/chromadb/db/mixins/sysdb.py", line 139, in get_tenant
    raise NotFoundError(f"Tenant {name} not found")
chromadb.db.base.NotFoundError: Tenant <TENANT_NAME> not found
INFO:     [11-07-2024 14:26:38] 10.244.1.165:59602 - "GET /api/v1/tenants/<TENANT_NAME> HTTP/1.1" 500

I updaed my dockerfile to:

ARG VECTOR_DB_ENGINE_VERSION
FROM chromadb/chroma:${VECTOR_DB_ENGINE_VERSION}

ENV IS_PERSISTENT=true
ENV PERSIST_DIRECTORY=/chroma/chroma
ENV ANONYMIZED_TELEMETRY=false
ENV CHROMA_SERVER_NOFILE=4096

ENV CHROMA_SERVER_AUTH_CREDENTIALS_PROVIDER="chromadb.auth.token.TokenConfigServerAuthCredentialsProvider"
ENV CHROMA_SERVER_AUTH_PROVIDER="chromadb.auth.token.TokenAuthServerProvider"
ENV CHROMA_CLIENT_AUTH_PROVIDER="chromadb.auth.token_authn.TokenAuthClientProvider"
ENV CHROMA_SERVER_AUTHN_PROVIDER="chromadb.auth.token_authn.TokenAuthenticationServerProvider"

ARG VECTOR_DB_HEADER_TOKEN_TRANSPORT
ENV CHROMA_SERVER_AUTH_TOKEN_TRANSPORT_HEADER=VECTOR_DB_HEADER_TOKEN_TRANSPORT
ENV CHROMA_AUTH_TOKEN_TRANSPORT_HEADER=VECTOR_DB_HEADER_TOKEN_TRANSPORT

ARG VECTOR_DB_HEADER_AUTH_CREDS
ENV CHROMA_SERVER_AUTH_CREDENTIALS=VECTOR_DB_HEADER_AUTH_CREDS
ENV CHROMA_CLIENT_AUTH_CREDENTIALS=VECTOR_DB_HEADER_AUTH_CREDS
ENV chroma_server_authn_credentials=VECTOR_DB_HEADER_AUTH_CREDS
ENV CHROMA_SERVER_AUTHN_CREDENTIALS=VECTOR_DB_HEADER_AUTH_CREDS

EXPOSE 8000

COPY chroma_log_config.yml /chroma/chromadb/log_config.yml

but im still able to 

curl http://localhost:8000/api/v1/collections
[]
tazarov commented 1 month ago

@leobenkel, this is weird: GET /api/v1/tenants/<TENANT_NAME> HTTP/1.1 it seems you're sending a request with a placeholder tenant. If you haven't created such a tenant, then the error is expected.

leobenkel commented 1 month ago

@leobenkel, this is weird: GET /api/v1/tenants/<TENANT_NAME> HTTP/1.1 it seems you're sending a request with a placeholder tenant. If you haven't created such a tenant, then the error is expected.

No no. I just redacted the log to not leak out a customer's name :)

leobenkel commented 1 month ago

@leobenkel, this is weird: GET /api/v1/tenants/<TENANT_NAME> HTTP/1.1 it seems you're sending a request with a placeholder tenant. If you haven't created such a tenant, then the error is expected.

Indeed I did not create the tenant. Which is my original question in this post haha. I am asking how to initialize tenant and database. With the posgresql container I just had to set some environment variables. But I didn't find the equivalent for chromadb.

leobenkel commented 1 month ago

@tazarov ?