search

piratar / wasa2il

Direct democracy system
https://x.piratar.is
Other
28 stars 16 forks source link

Pin dependencies #242

Closed tomage closed 2 years ago

tomage commented 2 years ago

Phew.. this one is a bit tricky.

Before I start.. I did a bit of tidying up in couple of related files too (see fist commit, if you want to see those changes separate).

I also changed some of the logic behind installing MySQL vs PostgreSQL packages.. That was a big headache to get right.

The idea is this:

Pin all python dependencies/packages to the exact version running in production right now.

The way I did it was:

  1. Go to production servers, and run pip freeze in the virtualenv running wasa2il.
  2. Take that list, and put into requirements.txt.freeze. Since prod is using mysql, I took the mysql dependency and put it into requirements-mysql.txt.freeze.
  3. Changed the make setup (well, the requirements targets) so they use the .freeze files to install dependencies. This makes it so that anyone checking out the project will install exactly the requirements that are on prod.
  4. Added couple of targets to help updating/managing these requirements.
  5. Tidied up requirements.txt, adding some commentary. This file ought be managed by humans.

The end result is:

  1. Slightly tweaked Makefile
  2. Updated requirements.txt
  3. New .freeze files that contain the actual packages to be installed.

The final change to the prod server should be:

  1. None at all!

After merging this, I'll copy the prod virtualenv, and then try make setup, to verify that nothing is installed, and nothing changes (pip doesn't have --dry-run option).

Testing

Tested pretty rigorously on my local machine (archlinux on Thinkpad), but also created the 2 vagrant boxes found in the vagrants/ directory, and they also came up fine.

Future work:

  1. Actually start to update some packages.
  2. Look into fully deprecating support for PostgreSQL (if we don't need it, don't support it!).
  3. Look into defaulting to MySQL (but allow SQLite for local development). This would still force developers to install some MySQL system packages to support the python MySQL client library installation.
  4. Look into converting to a more mature package system