Intermittent pip install failures make Windows tests flaky

[astrojuanlu]

Description

I have observed several times (see https://github.com/kedro-org/kedro-plugins/pull/171 and https://github.com/kedro-org/kedro-plugins/pull/180) that sometimes the Windows tests fail with the following message:

ERROR: Could not build wheels for import-linter

Which is hardly our fault, but worth looking into. For now, I have submitted a pull request upstream that migrates the metadata to static https://github.com/seddonym/import-linter/pull/169 which hopefully helps the author publish wheels and might mitigate this issue. But just in case it takes some time, here's the complete logs of a failing build and a successful one in case someone wants to investigate.

https://gist.github.com/astrojuanlu/62d78e4f075648c6842bb15ff412d1f4

[astrojuanlu]

Hopefully will soon stop being a problem https://github.com/seddonym/import-linter/issues/166

[seddonym]

I have just published a wheel for Import Linter 1.8.0. Could you let me know if this resolves your problem, and I will continue to do it for future releases?

[astrojuanlu]

Thanks a lot @seddonym! I'll keep an eye on the CI. Since this is an intermittent issue, I'll close it and if it reappears, I'll reopen.

[noklam]

This is amazing! Thanks @seddonym

[astrojuanlu]

Despite import-linter uploading wheels, the issue still occurs: https://app.circleci.com/pipelines/github/kedro-org/kedro/21627/workflows/1bfb0f1a-4d01-4bf4-b9d1-ffe71e5561e6/jobs/252183 (from gh-2568)

And the reason is that we're using an older version, 1.2.6

https://github.com/kedro-org/kedro/blob/653145771fa440a46b36d2c96c57a5997613da5e/test_requirements.txt#L20

[astrojuanlu]

This is happening again, and it's no longer related to import-linter, happens with other packages too https://app.circleci.com/pipelines/github/kedro-org/kedro/22850/workflows/31275040-c00a-482c-a780-0ccdc1c0b190/jobs/263096

  WARNING: Built wheel for parse is invalid: Wheel has unexpected file name: expected 'parse', got 'UNKNOWN'

For the record, that builder seems to be using pip 23.0.1.

Apparently we're not alone, see this report in the Mozilla issue tracker https://bugzilla.mozilla.org/show_bug.cgi?id=1732228, but looking at the patches, it's not clear to me what the root cause was https://hg.mozilla.org/mozilla-central/rev/705d211ab4df

I'd try using the latest pip, 23.1.2 at the moment, even though I don't see anything relevant in the pip issue tracker. And if it keeps failing, it's worth opening an issue there, or a discussion in https://discuss.python.org/c/packaging/14

[astrojuanlu]

Another one: https://app.circleci.com/pipelines/github/kedro-org/kedro/23662/workflows/86d82f9f-0b38-4b70-bb08-48d568fb66a3/jobs/271569

ERROR: Could not build wheels for psutil, which is required to install pyproject.toml-based projects

psutil 5.8.0 has abundant wheels https://pypi.org/project/psutil/5.8.0/#files

[astrojuanlu]

https://discuss.python.org/t/intermittent-issues-with-unknown-wheels-on-circleci-windows-builders/28570?u=astrojuanlu

[astrojuanlu]

Well, the workaround will be to avoid using sdists whenever possible 🙃

[astrojuanlu]

I haven't spotted this anymore on GitHub Actions, I'm closing for good