[Bug]: OpenShift Deployment starts without Prisma

abeltre1 commented 1 month ago

What happened?

Problem:

In this issue, the default litellm command within the LiteLLM container attempts to execute prisma generate within a directory facing two significant challenges that must be addressed on OpenShift for the command to successfully write to the directory:

Permissions: The target path, /usr/local/lib/python3.11/site-packages/prisma/, is mounted as read-only.
Overwriting: The file schema.prisma, already present at the target path, cannot be overwritten. This is due to the function call shutil.copy(data.schema_path, packaged_schema) attempting to overwrite schema.prisma.

Solution:

Build Without Prisma Schema: A straightforward solution would be to construct the upstream container without including the Prisma schema at the path /usr/local/lib/python3.11/site-packages/prisma/schema.prisma.
Rebuild with Adjusted Permissions and File Removal: A more involved solution entails using the upstream container as a base and then rebuilding the LiteLLM image with adjusted permissions to allow writing to the directory and removing the pre-existingschema.prismafile. This way we can execute litellm command can execute prisma generate without encountering issues.

It is worth highlighting that OpenShift permissions are stricter security policies.

Relevant log output

Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/prisma/generator/generator.py",
line 112, in run
self._on_request(request)
File "/usr/local/lib/python3.11/site-packages/prisma/generator/generator.py",
line 170, in _on_request
self.generate(data)
File "/usr/local/lib/python3.11/site-packages/prisma/generator/generator.py",
line 255, in generate
shutil.copy(data.schema_path, packaged_schema)
File "/usr/local/lib/python3.11/shutil.py", line 431, in copy
copyfile(src, dst, follow_symlinks=follow_symlinks)
File "/usr/local/lib/python3.11/shutil.py", line 258, in copyfile
with open(dst, 'wb') as fdst:
^^^^^^^^^^^^^^^
OSError: [Errno 30] Read-only file system: '/usr/local/lib/python3.11/site-packa
ges/prisma/schema.prisma'



### Twitter / LinkedIn details

_No response_

ishaan-jaff commented 1 month ago

@abeltre1 can you try this with our non root image - it's built to solve this issue: https://github.com/BerriAI/litellm/pkgs/container/litellm-non_root

abeltre1 commented 1 month ago

@abeltre1 can you try this with our non root image - it's built to solve this issue: https://github.com/BerriAI/litellm/pkgs/container/litellm-non_root

Hello @ishaan-jaff,

I switched to the non-root image as you suggested, but I encountered a PermissionError: [Errno 13] Permission denied: '/usr/local/lib/python3.11/site-packages/prisma/schema.prisma'. This type of permission issue is quite common in OpenShift due to its security model, which restricts write permissions for non-root users.

To address this, I modified the Dockerfile with the following workaround:

FROM ghcr.io/berriai/litellm-database:main-v1.43.18

USER 0

RUN chgrp -R 0 /usr/local/lib/python3.11/site-packages/prisma && \
    chmod -R g=u /usr/local/lib/python3.11/site-packages/prisma && \
    chmod -R +w /usr/local/lib/python3.11/site-packages/prisma

# Delete this file so it can be re-created with the correct permissions
# by whatever user OpenShift assigns at runtime.
RUN rm -rf /usr/local/lib/python3.11/site-packages/prisma/schema.prisma

ENTRYPOINT ["litellm"]
CMD ["--port", "4000"]

This workaround effectively resolves the issue. However, incorporating these changes into the upstream Litellm image could significantly benefit all users, especially those deploying in secured environments like OpenShift. It would enhance the out-of-the-box experience by ensuring compatibility with OpenShift's security policies without requiring manual adjustments.

Would it be possible for you to consider adding these adjustments to the Litellm upstream image? This change would not only streamline deployments in OpenShift but also align with best practices for container security by supporting non-root users.

Reference: Support arbitrary user ids

BerriAI / litellm