CircleCI-Public / python-orb

Common CircleCI tasks for the Python programming language.
https://circleci.com/developer/orbs/orb/circleci/python
MIT License
13 stars 37 forks source link

Cache ~/.local/bin when saving. #26

Closed jaqx0r closed 4 years ago

jaqx0r commented 4 years ago

If you install a python command with pip, then you get back the library, and no command, and then "pip install --user foo" gives you odd failures where it claims foo is in fact installed! But you can't see ~/.local/bin/foo and you're tearing your hair out.

Checklist

Motivation, issues

If you install a python command with pip, then you get back the library, and no command, and then "pip install --user foo" gives you odd failures where it claims foo is in fact installed! But you can't see ~/.local/bin/foo and you're tearing your hair out.

Description

Cache pip installed binaries as well as libraries to get a more complete record of the cached requirements.

FelicianoTech commented 4 years ago

My concern here is that ~/.local/bin/ is not a Python specific directory. If we automatically cache it in this orb's cache command, it may cache other items that might be there from other software. Do we want that? Can that cause a problem?

This actually also applies to the directory we're already caching, ~/.local/lib/.

jaqx0r commented 4 years ago

Yeah, that's right. So the orb should really be fetching the install path from pip and caching what pip thinks it has.

I don't know what that would take. In the meantime, the orb is broken, and I assume it's not being used for the case you suggest because of the environment orbs apply to.

On Wed, 29 Jan 2020, 05:16 Ricardo N Feliciano, notifications@github.com wrote:

My concern here is that ~/.local/bin/ is not a Python specific directory. If we automatically cache it in this orb's cache command, it may cache other items that might be there from other software. Do we want that? Can that cause a problem?

This actually also applies to the directory we're already caching, ~/.local/lib/.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/CircleCI-Public/python-orb/pull/26?email_source=notifications&email_token=AAXFX672476FW5GK3OZUCUTRABY6HA5CNFSM4KMPEXNKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKELHII#issuecomment-579384225, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAXFX67HNAUNWLFPV3AHZVTRABY6HANCNFSM4KMPEXNA .

jaqx0r commented 4 years ago

Actually I think it'd be easy to do correctly: instead of saving ~/.local/{bin,lib}, have an intermediate step that copies out the full manifest from pip into a temporary path or tarball, and cache that; and reverse that for load-cache.

On Wed, 29 Jan 2020 at 08:04, Jamie Wilkinson jaq@spacepants.org wrote:

Yeah, that's right. So the orb should really be fetching the install path from pip and caching what pip thinks it has.

I don't know what that would take. In the meantime, the orb is broken, and I assume it's not being used for the case you suggest because of the environment orbs apply to.

On Wed, 29 Jan 2020, 05:16 Ricardo N Feliciano, notifications@github.com wrote:

My concern here is that ~/.local/bin/ is not a Python specific directory. If we automatically cache it in this orb's cache command, it may cache other items that might be there from other software. Do we want that? Can that cause a problem?

This actually also applies to the directory we're already caching, ~/.local/lib/.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/CircleCI-Public/python-orb/pull/26?email_source=notifications&email_token=AAXFX672476FW5GK3OZUCUTRABY6HA5CNFSM4KMPEXNKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKELHII#issuecomment-579384225, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAXFX67HNAUNWLFPV3AHZVTRABY6HANCNFSM4KMPEXNA .

jaqx0r commented 4 years ago

I made us a bug to track this instead: #27

On Wed, 29 Jan 2020 at 09:49, Jamie Wilkinson jaq@spacepants.org wrote:

Actually I think it'd be easy to do correctly: instead of saving ~/.local/{bin,lib}, have an intermediate step that copies out the full manifest from pip into a temporary path or tarball, and cache that; and reverse that for load-cache.

On Wed, 29 Jan 2020 at 08:04, Jamie Wilkinson jaq@spacepants.org wrote:

Yeah, that's right. So the orb should really be fetching the install path from pip and caching what pip thinks it has.

I don't know what that would take. In the meantime, the orb is broken, and I assume it's not being used for the case you suggest because of the environment orbs apply to.

On Wed, 29 Jan 2020, 05:16 Ricardo N Feliciano, notifications@github.com wrote:

My concern here is that ~/.local/bin/ is not a Python specific directory. If we automatically cache it in this orb's cache command, it may cache other items that might be there from other software. Do we want that? Can that cause a problem?

This actually also applies to the directory we're already caching, ~/.local/lib/.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/CircleCI-Public/python-orb/pull/26?email_source=notifications&email_token=AAXFX672476FW5GK3OZUCUTRABY6HA5CNFSM4KMPEXNKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKELHII#issuecomment-579384225, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAXFX67HNAUNWLFPV3AHZVTRABY6HANCNFSM4KMPEXNA .