freedomofpress / securedrop-builder

Packaging logic for building SecureDrop-related Debian packages
GNU General Public License v3.0
7 stars 11 forks source link

Use poetry for bootstrap #478

Closed eloquence closed 8 months ago

eloquence commented 9 months ago

Resolves #468

This replaces our use of pip-compile with poetry to manage the dependencies for building other wheels, and updates the instructions accordingly.

To do so, it freezes all bootstrap dependencies at the current state, except for the ones we no longer need.

Context

Our bootstrap is quite fragile due to being so dated, and changes to individual dependencies can either cause breakage, changes to existing wheels, or both. Most notably, we cannot update to the latest Cython until we have removed our use of PyYAML (https://github.com/freedomofpress/securedrop-client/issues/1681), which is underway. Instead of opportunistic piecemeal updates, #464 proposes a full update of all bootstrap requirements, which I would suggest we do after we've removed PyYAML so we don't have to worry about that piece.

Testing

To test this PR, try rebuilding the bootstrap by following the instructions in the README, with a test dependency such as cowsay, while on this branch:

# Ensure you are running in a cleanly boostrapped virtual environment
rm -rf .venv
make install-deps
source .venv/bin/activate
# Update the workstation-bootstrap/pyproject.toml file with a new dependency
# For example, add `cowsay = "6.0.0"` to `pyproject.toml`
# Then update the lockfile:
poetry -C workstation-bootstrap/ lock
# Now we are ready to build updated wheels:
./scripts/build-sync-wheels --project workstation-bootstrap --pkg-dir ./workstation-bootstrap
# Once the new wheels are ready, we recreate our sha256sums:
./scripts/sync-sha256sums ./workstation-bootstrap
# Sign the list of sha256sums
gpg --armor --output workstation-bootstrap/sha256sums.txt.asc --detach-sig  workstation-bootstrap/sha256sums.txt
# We can even verify if we want
./scripts/verify-sha256sum-signature ./workstation-bootstrap/
# Update the build-requirements.txt file
./scripts/update-requirements --pkg-dir ./workstation-bootstrap/ --project workstation-bootstrap

Check if you encounter errors or unexpected results (git diff) along the way.