This PR adds a new image with the following differences w.r.t. the previous one:
Installs transformers latest release as of this commit's date, including Gemma 2 stable support for inference / training
Installs trl latest release as of this commit's date i.e. v0.9.6 (see #44)
Additionally, in top of the previous one aligns the formatting to use indent with 4 spaces instead of 2 spaces, since that's Docker's default for better readability, use double-quotes over single-quotes when defining the build arguments, and installs Python 3.10 as per the directory name, since it was under the py310 one, but installing Python 3.11
Open questions
Do we need to maintain both images before the release of the official / public one? Or is the latest i.e. the one within this PR, enough?
Should we have separate images for the "standard" stack i.e. transformers and another one for the LLM stack i.e. trl, or are we good having one with everything required? Asking since we have the transformers subdirectory and not sure if that's for pure readability or also for adding more frameworks eventually.
Do we prefer Python 3.11 over Python 3.10, or are we good with Python 3.10 for the moment? As a side-note, the current official / public DLCs that come with Python come with 3.10 (so maybe it's better to leave as is and install Python 3.10 as we're doing already)
Description
This PR adds a new image with the following differences w.r.t. the previous one:
transformerslatest release as of this commit's date, including Gemma 2 stable support for inference / trainingtrllatest release as of this commit's date i.e. v0.9.6 (see #44)Additionally, in top of the previous one aligns the formatting to use indent with 4 spaces instead of 2 spaces, since that's Docker's default for better readability, use double-quotes over single-quotes when defining the build arguments, and installs Python 3.10 as per the directory name, since it was under the
py310one, but installing Python 3.11Open questions
transformersand another one for the LLM stack i.e.trl, or are we good having one with everything required? Asking since we have thetransformerssubdirectory and not sure if that's for pure readability or also for adding more frameworks eventually.cc @philschmid