Management of dependencies between package `auto-nlp-workflow-lib` and image `huggingface-pytorch-gpu`

[cgawron]

The docker image huggingface-pytorch-gpu basically wraps the auto-nlp-workflow-lib package and allows trainings and deployments to be run.

The management of dependency is currently quite tricky:

• The Dockerfile starts with a pytorch image and installs some packages (like datasets and transformers) directly and adds auto-nlp-workflow-lib via pip install -e.
• The auto-nlp-workflow-lib package specifies dependencies in it's setup.py via install_requires. These dependencies include PyTorch, so it's easy to accidentally specify a different version here than provided by the pre-built image.
• To make matters worse, the package also contains a Pipfile and a Pipfile.lock (which are not used when building the container image).

@tnte Please comment if my understanding is incorrect ...

It would be preferable if each depency is specified only once or – if that is not possible – we should add some comments how these specifications are related to each other.

[cgawron]

All dependencies are now defined in the setup.py of auto-nlp-workflow-lib.