search

bcgov / von

Verifiable Organizations Network
https://digital.gov.bc.ca/digital-trust
Other
51 stars 22 forks source link

New version of pip breaks builds #359

Closed esune closed 3 years ago

esune commented 4 years ago

Starting October 2020 python builds using the latest version of pip started breaking. This is likely due to a new dependency resolution mechanism rolled out recently, as described here.

This issue is to track the work required to investigate and resolve the problem.

Affected projects (list updated as investigation/detection proceeds):

esune commented 4 years ago

Based on the deprecation timeline and the version of pip currently in our images (20.2.3) it looks like this may actually be an issue with the packages rather than pip itself, since it should default to the legacy behaviour we have been using until now.

Investigating further, but if this is confirmed pinning dependencies as-needed may be the way to go after all.

swcurran commented 4 years ago

This seems to me (and I could be totally wrong) to be a bad reason to pin dependencies. It would be good to chase down to root cause one of these to see what's happening.

esune commented 4 years ago

This looks like an issue with the transitive dependency, I posted in that repo and we can track progress isnce it seems a widespread problem: https://github.com/RonnyPfannschmidt/iniconfig/issues/17

For the time being I have pinned the dependency in aries-vcr so we can successfully build new versions, otherwise we have a broken pipeline.

About pinning dependencies: as usual, it depends.

It is generally best practice to lock all the dependencies used in a project to reduce issues like this. We are using a wide range of valid dependency versions, so if my environment had 1.0.1 cached it would not pull a newer version and the build would succeed. When building in OCP - or starting from scratch, the newest version of that dependency would be pulled and break the build. This problem is/was common in Node applications as well, and what has become best practice is to use lockfiles to pin EVERY dependency at a specific moment in time so that the build can reference the exact versions used when it was built the last time by someone handling those dependencies.

For python there is no built-in mechanism that I know that would do the same, pipenv is a package that could be added to achieve that, but it would also require us to update all of our repositories and possibly build pipelines if not supported out-of-the-box by s2i or explicitly using a Docker build.

I would see if the maintainers of that package - among others - come up with a fix otherwise we will need to come up with a plan to better manage dependencies ourselves.

swcurran commented 4 years ago

Good stuff -- thanks for chasing this down @esune -- I feel better.

esune commented 4 years ago

I unpinned the dependency as the new release of the affected package fixes things. Keeping this open since we will likely want to test the pip package resolution as well and come up with a plan for long-term maintenance of our projects (including aca-py).

esune commented 3 years ago

Closing since we are not seeing issues in our builds. New issues can be opened if we decide to manage dependencies differently, or new build errors start occurring.