BerriAI / litellm

Call all LLM APIs using the OpenAI format. Use Bedrock, Azure, OpenAI, Cohere, Anthropic, Ollama, Sagemaker, HuggingFace, Replicate (100+ LLMs)
https://docs.litellm.ai/docs/
Other
10.16k stars 1.13k forks source link

[Bug]: Helm-Chart / Can't run chart in root-restricted OpenShift Cluster #4456

Open freinold opened 3 days ago

freinold commented 3 days ago

What happened?

When trying to run the Helm chart in a root-restricted OpenShift (Red Hats Enterprise K8s) Cluster, prisma throws the following error: AssertionError: Target 'bin' directory does not exist

I already mounted the /.cache directory to a Volume to fix an error occurring while prisma dowloads its binaries, but I am not able to grant the Pod root access due to security restrictions enforced on the cluster.

I am also aware that this problem is at least known due to this comment in the values.yaml.

A possible workaround could be the installation of all needed prisma binaries during the build of the image, so that there are no root permissions needed at runtime. I see no general issue in this interfering with the current usage of the framework, especially since the schema is already frozen with the application version. Further this would follow the container best practices of Immutability and avoiding privileged containers.

I'm sadly not familiar with this framework at the time, but would be willing to help and contribute if someone could lead me in the right direction.

Relevant log output

Installing Prisma CLI
Traceback (most recent call last):
  File "/usr/local/bin/prisma", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prisma/cli/cli.py", line 39, in main
    sys.exit(prisma.run(args[1:]))
             ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prisma/cli/prisma.py", line 36, in run
    entrypoint = ensure_cached().entrypoint
                 ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prisma/cli/prisma.py", line 93, in ensure_cached
    proc = npm.run(
           ^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prisma/cli/_node.py", line 76, in run
    env=_update_path_env(
        ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prisma/cli/_node.py", line 298, in _update_path_env
    assert target_bin.exists(), 'Target `bin` directory does not exist'
AssertionError: Target `bin` directory does not exist
INFO:     Started server process [1]
INFO:     Waiting for application startup.
11:46:59 - LiteLLM Proxy:ERROR: key_management_endpoints.py:856 - litellm.proxy.proxy_server.generate_key_helper_fn(): Exception occured - Not connected to the query engine
Task exception was never retrieved
future: <Task finished name='Task-9' coro=<generate_key_helper_fn() done, defined at /usr/local/lib/python3.11/site-packages/litellm/proxy/management_endpoints/key_management_endpoints.py:668> exception=HTTPException(status_code=500, detail={'error': 'Internal Server Error.'})>
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/litellm/proxy/management_endpoints/key_management_endpoints.py", line 836, in generate_key_helper_fn
    user_row = await prisma_client.update_data(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/backoff/_async.py", line 151, in retry
    ret = await target(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/litellm/proxy/utils.py", line 1675, in update_data
    raise e
  File "/usr/local/lib/python3.11/site-packages/litellm/proxy/utils.py", line 1513, in update_data
    update_user_row = await self.db.litellm_usertable.upsert(
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prisma/actions.py", line 5864, in upsert
    resp = await self._client._execute(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prisma/client.py", line 525, in _execute
    return await self._engine.query(builder.build(), tx_id=self._tx_id)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prisma/engine/query.py", line 244, in query
    return await self.request(
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prisma/engine/http.py", line 97, in request
    raise errors.NotConnectedError('Not connected to the query engine')
prisma.engine.errors.NotConnectedError: Not connected to the query engine

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/litellm/proxy/management_endpoints/key_management_endpoints.py", line 864, in generate_key_helper_fn
    raise HTTPException(
fastapi.exceptions.HTTPException: 500: {'error': 'Internal Server Error.'}
ERROR:    Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/litellm/proxy/utils.py", line 772, in check_view_exists
    await self.db.query_raw(
  File "/usr/local/lib/python3.11/site-packages/prisma/client.py", line 421, in query_raw
    resp = await self._execute(
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prisma/client.py", line 525, in _execute
    return await self._engine.query(builder.build(), tx_id=self._tx_id)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prisma/engine/query.py", line 244, in query
    return await self.request(
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prisma/engine/http.py", line 97, in request
    raise errors.NotConnectedError('Not connected to the query engine')
prisma.engine.errors.NotConnectedError: Not connected to the query engine

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 732, in lifespan
    async with self.lifespan_context(app) as maybe_state:
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 608, in __aenter__
    await self._router.startup()
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 709, in startup
    await handler()
  File "/usr/local/lib/python3.11/site-packages/litellm/proxy/proxy_server.py", line 2768, in startup_event
    create_view_response = await prisma_client.check_view_exists()
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/backoff/_async.py", line 151, in retry
    ret = await target(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/litellm/proxy/utils.py", line 778, in check_view_exists
    value = await self.health_check()
            ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/litellm/proxy/utils.py", line 1856, in health_check
    raise e
  File "/usr/local/lib/python3.11/site-packages/litellm/proxy/utils.py", line 1838, in health_check
    response = await self.db.query_raw(sql_query)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prisma/client.py", line 421, in query_raw
    resp = await self._execute(
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prisma/client.py", line 525, in _execute
    return await self._engine.query(builder.build(), tx_id=self._tx_id)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prisma/engine/query.py", line 244, in query
    return await self.request(
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prisma/engine/http.py", line 97, in request
    raise errors.NotConnectedError('Not connected to the query engine')
prisma.engine.errors.NotConnectedError: Not connected to the query engine

ERROR:    Application startup failed. Exiting.

Twitter / LinkedIn details

@f_reinold / www.linkedin.com/in/fabian-reinold