🔧 Add direct dependencies file: requirements.in

Here is a proposal to make it easier to create and add dependencies, and to be alerted to security issues in the packages we use.

Why?

The goal is:

To be automatically notified of security vulnerabilities in the project's Python packages. The GitHub Dependabot (dependencies bot) can automatically search and notify us of security vulnerabilities in the packages used directly and indirectly by our project (provided they are all listed in requirements.txt).
A separate file for direct dependencies
To make adding a top-level (i.e. direct) dependency easier.
To distinguish between:
- direct dependencies in a new file requirements.in.
- all the dependencies (including indirect dependencies aka. sub-dependencies) in requirements.txt

Today, when using a single file with all dependencies (requirements.txt), I find it difficult to spot the direct dependencies (sphinx, sphinx-copybutton, sphinx-rtd-theme, myst-parser) in an ocean of indirect dependencies. So, I suggest using 2 files.

How?

split the current dependencies file requirements.txt to have 2 files:

requirements.in containing only the direct dependencies

sphinx==5.3.0
sphinx-copybutton==0.5.2
sphinx-rtd-theme==1.2.0
myst-parser==0.18.1

requirements.txt containing the exhaustive list of all dependencies (direct and indirect). Today it does not contain the latter.

#
# This file is autogenerated by pip-compile with Python 3.12
# by the following command:
#
#    pip-compile
#
alabaster==0.7.16
# via sphinx
babel==2.16.0
# via sphinx
certifi==2024.8.30
# via requests
charset-normalizer==3.3.2
# via requests
docutils==0.18.1
# via
#   myst-parser
#   sphinx
#   sphinx-rtd-theme
idna==3.10
# via requests
imagesize==1.4.1
# via sphinx
jinja2==3.1.4
# via
#   myst-parser
#   sphinx
markdown-it-py==2.2.0
# via
#   mdit-py-plugins
#   myst-parser
markupsafe==2.1.5
# via jinja2
mdit-py-plugins==0.3.5
# via myst-parser
mdurl==0.1.2
# via markdown-it-py
myst-parser==0.18.1
# via -r requirements.in
packaging==24.1
# via sphinx
pygments==2.18.0
# via sphinx
pyyaml==6.0.2
# via myst-parser
requests==2.32.3
# via sphinx
snowballstemmer==2.2.0
# via sphinx
sphinx==5.3.0
# via
#   -r requirements.in
#   myst-parser
#   sphinx-copybutton
#   sphinx-rtd-theme
#   sphinxcontrib-jquery
sphinx-copybutton==0.5.2
# via -r requirements.in
sphinx-rtd-theme==1.2.0
# via -r requirements.in
sphinxcontrib-applehelp==2.0.0
# via sphinx
sphinxcontrib-devhelp==2.0.0
# via sphinx
sphinxcontrib-htmlhelp==2.1.0
# via sphinx
sphinxcontrib-jquery==4.1
# via sphinx-rtd-theme
sphinxcontrib-jsmath==1.0.1
# via sphinx
sphinxcontrib-qthelp==2.0.0
# via sphinx
sphinxcontrib-serializinghtml==2.0.0
# via sphinx
typing-extensions==4.12.2
# via myst-parser
urllib3==2.2.3
# via requests

pip-compile lists below each dependencies its sub-dependencies.

Use pip-compile (from the pip-tools package) to generate requirements.txt from requirements.in.

pip-compile reads a source file requirements.in to generate a requirements.txt with all the dependencies. It resolves and pins the dependencies to the ad-hoc version, ensuring reproducibility. It only focuses on dependencies declared in the source file.

pip freeze uses all packages installed in the virtual environment.

Install

We need to install pip-tools first to use pip-compile:

python -m pip install pip-tools

Usage

Using pip-compile:

pip-compile
python -m pip install -r requirements.txt

pip-compile only looks at the source file (requirements/in), whereas pip freeze looks at what is currently installed in the virtual environment.

Previously, with pip:

# Assuming the `venv` virtual environment has already been created and activated
# python -m venv venv
# source venv/bin/activate

# Remove all installed dependencies/packages
python -m pip freeze --exclude-editable | xargs python -m pip  uninstall -y

# Install the project's packages listed in `requirements.in`
python -m pip install -r requirements.in

# List the pinned project packages (name + version) in `requirements.txt`
python -m pip freeze > requirements.txt

# Install dependencies
python -m pip install -r requirements.txt

nightscout / trio-docs