search

mk-fg / pretty-yaml

PyYAML-based module to produce a bit more pretty and readable YAML-serialized data
Do What The F*ck You Want To Public License
135 stars 29 forks source link

Pin PyYAML 5.1 version in order to support deterministic builds of upstream projects #28

Closed borisuvarov closed 5 years ago

borisuvarov commented 5 years ago

Hi @mk-fg, could you please take a look? I think this change is pretty useful, since a few days ago an unpinned PyYAML from pyaml crashed a pretty important production system. :-) Thanks in advance! Please let me know if I should adjust/change this PR somehow.

mk-fg commented 5 years ago

I think this will cause pip to install PyYAML==5.1 from here to eternity, instead of latest version, correct?

While I see how it might be desirable now on specific production setup, this version pin would have to be maintained and updated whenever new PyYAML comes out (which I'm unlikely to remember and don't want to bother with), otherwise it'd be forcing pip to install old version for no good reason.

Can see how such non-determinism indeed can break upstream projects, but if they require specific PyYAML version, it's a bug that PyYAML isn't pinned there, and if older or newer PyYAML version breaks this module - let me know, I'll either make it compatible (for new versions) or add "PyYAML>x" to setup.py (for older versions), as it'd be a bug here.

Also, I'd suggest that you might want to use requrements.txt in replicated production setup, where every installed python package is not only pinned, but also checksummed, which will prevent such non-determinism, and seem to be made exactly for this purpose.

mk-fg commented 5 years ago

might want to use requrements.txt

Alternatively, if it's done a hundred times, maybe just roll all these into signed wheels and install these as a bundle? To avoid straining public pypi mirrors, if nothing else.

borisuvarov commented 5 years ago

@mk-fg Thanks, this makes sense. We're using pipenv and its Pipfile.lock for all the new projects, but some legacy stuff still relies on pip + requirements.txt, and someone forgot to dump a full output of pip freeze into the requirements.txt.