Upgrade instructions for dependencies

Flix6x commented 1 year ago

While upgrading a server from FlexMeasures 0.14 to 0.15, I ran into a ModuleNotFoundError: No module named 'email_validator.deliverability', which can be resolved by upgrading the email-validator package to the version specified in any one of the requirements app.txt files: pip install email-validator==2.0.0.post2.

I then realized that a pip install flexmeasures -U does not update any other packages per se, and so this server would not run on the same dependencies as specified in our requirements.txt for Python 3.8 (which this server happens to be running). My way forward was to do a pip install flexmeasures -U --upgrade-strategy eager to force all of FM's dependencies to upgrade to the latest compatible version.

Perhaps our changelog's upgrade instructions should plainly mention this command (for each version, or just once at the top), and hint at the possibility and consequences of using the eager upgrade strategy?

nhoening commented 1 year ago

I believe I should have changed app.in to say we require a version >2 of that package since I updated our code which imports it. That would probably have helped the usual pip install command to get it. Maybe that is a candidate for the mailing 0.15.1 patch release.

Btw, a first installation would grab the latest dependencies and work fine. This is about upgrading, as the title correctly states.

nhoening commented 10 months ago

Are we doing better with this, as we have ci/update-packages.sh? It can distinguish between upgrade (use upgrade parameter) or freeze (the default).

Flix6x commented 8 months ago

I should have changed app.in to say we require a version >2 of that package since I updated our code which imports it. That would probably have helped the usual pip install command to get it.

I think you're right. Then perhaps we could suggest that existing hosts using pip installation have two options:

Use pip install flexmeasures -U to only upgrade dependencies that are no longer compatible with FlexMeasures, e.g. due to a new minimum requirement (NB these dependencies will be updated to the latest compatible version).
Use pip install flexmeasures -U --upgrade-strategy eager to get all latest dependencies that are compatible with FlexMeasures, to our knowledge (NB we have automated tooling in place to check whether new releases of dependencies are compatible, but we don't release patch versions automatically).

Are we doing better with this, as we have ci/update-packages.sh? It can distinguish between upgrade (use upgrade parameter) or freeze (the default).

But this is not a script for hosts using pip installation, right? Doesn't it only update .txt requirements, while pip is using the .in requirements?

nhoening commented 1 month ago

Doesn't ci/update-packages.sh only update .txt requirements, while pip is using the .in requirements?

No, it uses make upgrade-deps or make freeze-deps, which both use pip-compile where the .in files are read in.

FlexMeasures / flexmeasures

Upgrade instructions for dependencies #820