Dockerfiles should use pyproject.toml to install optional dependencies

[jhale]

Describe new/missing feature

We currently attempt to include optional dependencies in our pyproject.toml file

https://github.com/FEniCS/dolfinx/blob/main/python/pyproject.toml

But this remains untested as the CI image builds manually type optional dependencies

https://github.com/FEniCS/dolfinx/blob/main/docker/Dockerfile.test-env#L156

There is some discussion of this issue here with respect to adding e.g. --only-deps option to pip install: https://github.com/pypa/pip/issues/11440

Suggested user interface

Not applicable.

[jhale]

pip-tools seems to have support for creating requirements.txt files that can then be fed into pip.

https://github.com/jazzband/pip-tools

[garth-wells]

How about we slim down the images and let the CI runner install Python optional dependencies, documentation dependencies, etc. This makes testing, pinning and un-pinning versions much easier

[francesco-ballarin]

How about we slim down the images and let the CI runner install Python optional dependencies, documentation dependencies, etc. This makes testing, pinning and un-pinning versions much easier

:+1::+1::+1:

The only possible downside I see is that, if pinning was required, one would have to do that several times, for instance:

• in the CI runner of dolfinx,
• (in the CI runner of e.g. basix or ffcx, if the same pin were needed for them too)
• in the end user dockerfile

[jhale]

@francesco-ballarin The pyproject.toml would be the single source for specifying light Python optional requirements. We're already doing this in e.g. Basix and it works well see e.g. https://github.com/FEniCS/basix/blob/main/.github/workflows/pythonapp.yml#L42 https://github.com/FEniCS/basix/blob/main/pyproject.toml#L28

petsc4py, mpi4py and numpy and the other heavy packages will be built inside the Dockerfile as they are now.