Use pyproject.toml instead of setup.cfg, pytest.ini & requirements.txt and other Python related config files

[stv0g]

Modern Python is slowly adopting pyproject.toml as a single place to define a Python package and related tool settings like black for formatting, flake8 for linting and pytest for testing.

Lets try to move as much of the Python-related project configuration to this file and unclutter the root of the repo.

This PR does:

• Merge requirements*.txt, pytest.ini and setup.cfg files into pyproject.toml
• Update shell scripts and Dockerfiles to use pyproject.toml
• Introduce a pre-commit configuration file
• Add a GitHub actions workflow for running pre-commit also in the CI
• Apply/fix pre-commit checks to all files in the repo:
  • Harmonize file endings (single empty line)
  • Remove trailing whitespaces
  • Apply formatting of all Python code (also in Jupyter notebooks)
• Remove obsolete CI jobs from .gitlab-ci.yml

[stv0g]

I am interested in your opinions @leonardocarreras and @m-mirz. What do you think about consolidate all the Python related project files in pyproject.toml

[m-mirz]

Good idea 👍

[leonardocarreras]

I think it is a great idea 🙌🏻

[stv0g]

This is ready for review :)

I've also added pre-commit as a tool to perform several checks against changes which are staged for commit. Such as removing trailing whitespaces, checking JSON, YAML and XML syntax and Python / C++ code formatting.

While doing so, I realized that much of the Python code was ugly formatted. We are now using the de-facto standard Python formatter Black. It even can format Python code with in Jupyter notebooks :)

[stv0g]

@m-mirz I guess the SonarCloud warnings are false-positives. Its at least not code which I've touched.

[stv0g]

Hi @martinmoraga @leonardocarreras @dinkelbachjan,

Markus has reviewed and approved this change. However, we would like to find more approvals before going ahead and merging it.

Do you have a second to look at this change?

[leonardocarreras]

Side topic: Apparently, the more changes are here, the more Sonar complains and demands higher quality from reformatted existing code...

[sonarcloud[bot]]

Quality Gate failed

Failed conditions
5 Security Hotspots
E Security Review Rating on New Code (required ≥ A)
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarCloud

Catch issues before they fail your Quality Gate with our IDE extension SonarLint

[m-mirz]

@stv0g @leonardocarreras I think this can be merged after the question regarding pip install have been resolved.