CircleCI-Public / python-orb

Common CircleCI tasks for the Python programming language.
https://circleci.com/developer/orbs/orb/circleci/python
MIT License
13 stars 37 forks source link

Changing python versions when restoring cache breaks builds #51

Closed dsayling closed 4 years ago

dsayling commented 4 years ago

Orb version:

1.0.0+

What happened:

TLDR: The cache created in the python orb contains libraries and binaries that are stored or linked to a path that contains the python version, which causes an error in builds where a python version is updated.

The python orb caches the site-packages and the entire virtualenv (including the binaries and libraries), if using pipenv or poetry. For example, after creating a virtualenv you'll see that the python binary for the virtualenv titled "project" is still linked to the global binary.

(project) circleci@5a360a6f5121:~/project$ readlink -f $(which python)
/home/circleci/.pyenv/versions/3.8.5/bin/python3.8

If using pip-globally, we can show that the site-packages are linked to a path with a version as well

circleci@5a360a6f5121:~/project$ python -c "import setuptools; print(setuptools.__path__)"
['/home/circleci/.pyenv/versions/3.8.5/lib/python3.8/site-packages/setuptools']

Because of this, when you change the python version and don't update a dependency file, any restored cache will be linked to original python version and will cause the build to fail.

Expected behavior:

When the python version on the system changes, the orb should not restore from the existing cache.

Additional Information:

Work arounds:

There are a few work arounds to this while a fix is investigated.

  1. Make sure that your PIpfile or pyproject.toml specifies the full python version inside and regenerate the lock file. This will ensure the cache doesn't reload this time.
  2. Update the cache-version parameter on the command or job. This will ensure the cache doesn't reload this time.
  3. Ensure the tag parameter is set in the executor with the full version. The list of tags is available here.
          executor:
            name: python/default
            tag: 3.8.6