[bug] Guardrails Hub Install Path Mismatch with PDM Environment in Docker

[jairNeto]

Describe the bug

When using PDM for library management in a Dockerfile, installing guardrails from the hub with pdm run guardrails hub install hub://guardrails/valid_choices installs the package in a different path than other libraries managed by PDM. This results in an ImportError when trying to import the installed guardrails.

To Reproduce

1. Use the following Dockerfile setup:

   
   FROM python:3.11-slim

WORKDIR /app

COPY pyproject.toml pdm.lock ./

RUN pip install pdm && mkdir pypackages && pdm sync --prod --no-editable RUN pdm run guardrails hub install hub://guardrails/valid_choices

CMD ["pdm", "run", "python", "your_script.py"]

2. Attempt to import the installed guardrails in the Python script:

`from guardrails.hub import ValidChoices`

**Expected behavior**

The guardrails should be installed in the same path as other libraries managed by PDM, allowing for successful import without any additional steps.

**Library version:**

0.4.1

**Additional context**

The workaround to fix this issue involves manually moving the installed guardrails to the correct path:

RUN pip install pdm && mkdir pypackages && pdm sync --prod --no-editable RUN pdm run guardrails hub install hub://guardrails/valid_choices RUN mv /usr/local/lib/python3.11/site-packages/guardrails/hub/* /app/pypackages/3.11/lib/guardrails/hub/

This issue arises because the guardrails installed via pdm run guardrails hub install are placed in /usr/local/lib/python3.11/site-packages/guardrails/hub/ instead of /app/__pypackages__/3.11/lib/guardrails/hub/.

[CalebCourier]

Taking a look

[CalebCourier]

@jairNeto I made some progress on this today.

First a little background: The reason the hub validators are installed at /usr/local/lib/python3.11/site-packages/... is because the cli uses the system executable for pip to perform the installation. The discrepancy you're seeing is because pdm does not follow the same pattern as other package managers (poetry, pip, etc.).

However there is a way around this. If you use a virtual environment, pdm will utilize the same standard directories as other package managers since they are defined by the virtual environment backend (venv, conda, etc.). Below is an example docker file and test script that shows using venv for this.

Dockerfile

FROM python:3.11-slim

WORKDIR /app

# create the virtual environment
RUN python3 -m venv /opt/venv

# Enable the virtual environment
ENV PATH="/opt/venv/bin:$PATH"

COPY pyproject.toml pdm.lock test.py ./

# Install git
RUN apt-get clean
RUN apt-get update
RUN apt-get install -y git

RUN pip install pdm
# Tell pdm to use the virtual environment
RUN pdm use -f /opt/venv
RUN pdm sync --prod --no-editable
RUN pdm run guardrails hub install hub://guardrails/valid_choices

CMD ["pdm", "run", "python", "test.py"]

test.py

from guardrails.validator_base import PassResult, FailResult
from guardrails.hub import ValidChoices

validator = ValidChoices(choices=["a", "b", "c"])

result = validator.validate("a", {})
assert isinstance(result, PassResult)

result = validator.validate("d", {})
assert isinstance(result, FailResult)

print("All good! Exiting...")

[jairNeto]

All good @CalebCourier ! Thanks for the speed in the response👏👏👏