search

nat-n / poethepoet

A task runner that works well with poetry.
https://poethepoet.natn.io/
MIT License
1.4k stars 58 forks source link

Version 0.16.1 causes path issues on Windows in Github CI #88

Closed Dr-Irv closed 1 year ago

Dr-Irv commented 2 years ago

In the pandas-stubs project, we use poetry and poe in our CI. It started failing a couple of days ago, and after many hours of debugging between members of the core team, we were able to fix the problem by pinning poethepoet to version 0.16.0.

Here's an example log where from within our test scripts, we print out where python, where poe and where poetry and the value of sys.executable:

2022-09-01T22:25:22.1671090Z C:\Users\runneradmin\AppData\Local\pypoetry\Cache\virtualenvs\pandas-stubs-rIU6jvyd-py3.10\Scripts\python.exe
2022-09-01T22:25:22.1733214Z C:\hostedtoolcache\windows\Python\3.10.6\x64\python.exe
2022-09-01T22:25:22.1806760Z C:\hostedtoolcache\windows\Python\3.9.13\x64\python.exe
2022-09-01T22:25:22.2683021Z C:\Users\runneradmin\AppData\Local\pypoetry\Cache\virtualenvs\pandas-stubs-rIU6jvyd-py3.10\Scripts\poe.exe
2022-09-01T22:25:22.3708439Z C:\hostedtoolcache\windows\Python\3.10.6\x64\Scripts\poetry.exe
2022-09-01T22:25:22.4510595Z sys.executable  C:\hostedtoolcache\windows\Python\3.10.6\x64\python.exe

Note that sys.executable is the system python.

If we pin poethepoet to version 0.16.0, then everything is fine. Here's the same output:

2022-09-01T22:51:14.4885350Z C:\Users\runneradmin\AppData\Local\pypoetry\Cache\virtualenvs\pandas-stubs-rIU6jvyd-py3.10\Scripts\python.exe
2022-09-01T22:51:14.4947352Z C:\hostedtoolcache\windows\Python\3.10.6\x64\python.exe
2022-09-01T22:51:14.5015606Z C:\hostedtoolcache\windows\Python\3.9.13\x64\python.exe
2022-09-01T22:51:14.5797657Z C:\Users\runneradmin\AppData\Local\pypoetry\Cache\virtualenvs\pandas-stubs-rIU6jvyd-py3.10\Scripts\poe.exe
2022-09-01T22:51:14.6671531Z C:\hostedtoolcache\windows\Python\3.10.6\x64\Scripts\poetry.exe
2022-09-01T22:51:34.0166914Z sys.executable  C:\Users\runneradmin\AppData\Local\pypoetry\Cache\virtualenvs\pandas-stubs-rIU6jvyd-py3.10\Scripts\python.exe

Note that sys.executable is the python that is in the virtual environment when we use 0.16.0.

Work on this issue for us can be found in this PR: https://github.com/pandas-dev/pandas-stubs/pull/247 where we kept pinning releases and finally figured out that version 0.16.1 was the culprit. Our final fix was the pinning to version 0.16.0: https://github.com/pandas-dev/pandas-stubs/pull/249

nat-n commented 2 years ago

Hi @Dr-Irv,

Thanks very much for reporting this and for the efforts of you and your team to investigate the issue.

Apologies for the hassle, as soon as I have time I'll publish 0.16.2 to revert the changes in 0.16.1 while I investigate the root cause.

Could you also help me with a couple of details?

Dr-Irv commented 2 years ago

  • Does this problem only manifest on windows? Yes.

  • You mentioned this was happening in GitHub actions, could you perhaps give me a pointer on what repo/commit I would need to fork to replicate the issue?

The repo is https://github.com/pandas-dev/pandas-stubs . If you checkout tag v1.4.3.220829 and run it as a GH action, it should fail because it will install poethepoet 0.16.1. If you edit the line https://github.com/pandas-dev/pandas-stubs/blob/fa84c4c6f4663ce7651c475d39dd3d73aa5967b1/pyproject.toml#L42

to pin poethepoet to 0.16.0, the action will succeed.

nat-n commented 1 year ago

@Dr-Irv Thanks again for the bug report.

I finally got around to setting up a windows VM to properly debug and get to the bottom of this and 0.16.5 is the result. It includes all the changes from the broken version (which I had reverted) but doesn't break your CI, see: https://github.com/nat-n/pandas-stubs/commit/e76ae4c2dda8f99d60051971bf40303df9679733