Open zerolab opened 7 months ago
Follow up from @nimasmi in https://github.com/wagtail/wagtail-ai/issues/43#issuecomment-1860677558
This is the Dockerfile written the way I naively approached it. It's a work in progress, but may save some time if we want to continue with this approach.
FROM python:3.11-bookworm
# Install dependencies in a virtualenv
ENV VIRTUAL_ENV=/venv
RUN useradd vector --create-home && mkdir /app $VIRTUAL_ENV && chown vector /app $VIRTUAL_ENV
WORKDIR /app
ENV PATH=$VIRTUAL_ENV/bin:$PATH \
PYTHONUNBUFFERED=1
COPY . .
USER vector
RUN python -m venv $VIRTUAL_ENV
RUN pip install flit
RUN python -m flit install
RUN pip install tox
RUN tox
On the other hand, the Tox project recommends starting from the 31z4/tox
image. See Running within a Docker Container.
FROM 31z4/tox
USER root
RUN set -eux; \
apt-get update; \
DEBIAN_FRONTEND=noninteractive \
apt-get install -y --no-install-recommends \
python3.12 \
postgresql; \
rm -rf /var/lib/apt/lists/*
COPY --chown=tox . .
USER tox
From @nimasmi in https://github.com/wagtail/wagtail-ai/issues/43#issuecomment-1860660722
I had trouble running this at first. A lot of it stems from working with tox and flit. It's fine to use new tools, but in an agency that has an established stack and toolchain, expect to have to provide extra documentation for unfamiliar workflows.
Aside: I appreciate the current package architecture is not the fault of the current project team, and that it's a legacy of https://github.com/wagtail/cookiecutter-wagtail-package.
Setup:
tox
on a fresh repo didn't work, because it expects a system installation of Postgres, which I generally prefer not to have. I changedpsycopg2
topsycopg2-binary
in tox.ini, and this got me further through the tox Postgres environment build step, though it still didn't complete running the tests.dj_database_url
as a dependency; this is also mentioned in discussion on https://github.com/wagtail/wagtail-vector-index/pull/15, though not fixed by that PR.Python 3.12
None of the tox tests run for me in the
py312
environment. The issue I got wasbut since 3.12 is in alpha, I sort of ignored this one.
Changing code, and running tests
Firstly flit needed to be installed with the
-s
option. This documentation change was also fixed in https://github.com/wagtail/wagtail-vector-index/pull/15.When I modify the code, I need to run
tox -e py311-django4.2-wagtail5.2-postgres -r
(see note below). This takes 1 m 20 s, which is pretty tedious when you're checking a one-line change. If you are only modifying the tests, the-r
(recreate) flag is not necessary, but even then the tests take 20 s. Is there a way to run tests more quickly?Note: I stuck with the 3.11 environment because of the 3.12 failures, and the SQLite issue below, but also because one environment is quicker to test than four.
Databases and containers
I ran tests on PostgreSQL (due to the flit
-s
and tox-r
issues, at first I believed the tests were skipped, or returning false positives, on SQLite). I don't have Postgres installed on my machine, and always run it in containers or VMs when necessary.I started therefore trying to add a Dockerfile for this project. I got part way there, aiming for a Docker container to run the whole project, and contain multiple Pythons, so tox could do its thing inside that container. At that point I found out that the tox project recommends doing things differently: run a tox container, and let it manage the Python environments, I think. I'm not sure there how you would manage other dependencies such as a Postgres container, though.
Eventually @tm-kn pointed me at https://github.com/wagtail/wagtail-vector-index/issues/17, which was sufficient, so I didn't persist with getting the Dockerfiles working. I will post two Dockerfile examples in a separate update, just for discussion.
In general, though, there is so much configuration necessary that Docker might be handy, or at least we could do with better guidance for how to test locally.
Originally posted by @nimasmi in https://github.com/wagtail/wagtail-ai/issues/43#issuecomment-1860660722