Define dependency tracking workflow

[nidico]

We currently track Adhocracy dependencies in two places - setup.py and versions.cfg. @joka's understanding of this, which I find reasonable, is the following:

1. Direct dependencies are manually tracked in setup.py. This should only include minimal versions if we know that we depend on a certain feature / bug fix which was included in that version. And this should only include maximal versions if we know that the library breaks our code in a certain version.
2. A known-good set of all direct and indirect dependencies is maintained in versions.cfg. This makes sure that deployments always receive versions of dependencies which have been tested as good, and avoids breakage through new versions in PyPI.

How to identify min / max versions and update both files:

• An honest definition of minimal versions of direct dependencies is rather hard in my opinion, as it involves digging through the history of dependencies to find out at which point a certain feature has been introduced. As this is too much work, let's just put in what we guess or leave it open where appropriate - this can still be refined at a later point.
• To update versions.cfg, delete it and rerun buildout.
  • If the (automatic and manual) testsuite is still happy, set the depencies (see last lines of buildout output if show-picked-versions (buildout 2) or buildout.dumppickedversions (buildout 1) is enabled) as the new versions.cfg.
  • If not, identify the breakage and either fix it in our code to make it compatible with the new version of that dependency, or put it as a maximum version boundary in setup.py.

Is this fine for everyone, or does anybody has other ideas? If we can agree on that, let's dig through the dependencies once and clean it up, as the current minimum settings are rather arbitrary set.

[joka]

I would not put to much effort in finding minimal versions. If you make a fresh installation without version.cfg you get the latest versions anyway.

Instead of removing version.cfg its more handy to run "bin/checkversions -l 2 " or "bin/checkversions -l 1 " . Then you see exactly what is updated. Note: the checkervsions part is missing the adhocracy egg at the moment.

To compile/require Python 2.7 I make a new pull request, to get more feedback.

[joka]

Note: Our buildout runs in non-newest mode to make it faster. If you want newest mode run "bin/buildout -n".

[nidico]

Instead of removing version.cfg its more handy to run "bin/checkversions -v -l 2 > versions.cfg" or "bin/checkversions -v -l 1 > versions.cfg" . Then you see exactly what is updated.

Okay, if only updated that way, the current versions.cfg file always contains information about which eggs have been updated in the latest commit of this file. Well, why not.

My wish here is that the eggs shouldn't be moved around all the time in order to make commits easier readable.

To compile/require Python 2.7 I make a new pull request, to get more feedback.

Okay.

Note: Our buildout runs in non-newest mode to make it faster. If you want newest mode run "bin/buildout -n".

This is alright and shouldn't make no difference at all, as all dependencies should be pinned in versions.cfg anyway. If you want to upgrade stuff, you can either overwrite versions.cfg through a checkversions run as above (no need for -n then) or remove versions.cfg and run bin/buildout -n.

[joka]

We all agree its bad to maintain pinned versions in setup.py and versions.cfg.

I think its fine to add minimal versions in setup.py if we rely on a newly added features. But I still don't understand why we need version pinning and max versions in setup.py at all. If you want to support non buildout installations, we could generate an know good pip requirements file.

@phihag what you point in this?

[joka]

In https://github.com/liqd/adhocracy/pull/321 I added documentation how to maintain dependency versions.

[nidico]

But I still don't understand why we need version pinning and max versions in setup.py at all.

Max versions in setup.py only if we know something breaks, as written in the description, and as you wrote in maintain_versions.cfg.

If you want to support non buildout installations, we could generate an know good pip requirements file.

No need to do this IMO.

In #321 I added documentation how to maintain dependency versions.

Fine.

[joka]

Max versions in setup.py only if we know something breaks, as written in the description, and as you wrote in maintain_versions.cfg.

oki