search

pex-tool / pex

A tool for generating .pex (Python EXecutable) files, lock files and venvs.
https://docs.pex-tool.org
Apache License 2.0
2.49k stars 254 forks source link

Handle safe_copy failing to hardlink on Windows due to too many existing links #2394

Open akrieger opened 3 months ago

akrieger commented 3 months ago

Windows does support hard links, and os.link() supports Windows as of Python 3.2. There is a specific error that can be thrown on Windows when you try to create too many hardlinks to a file. Handle that case explicitly in its own test. It's more precise than trying to infer an errno for it which may be mapped to by various kinds of errors.

jsirois commented 3 months ago

@akrieger thanks for the effort here. Much more is required however. In short, Pex does not support Windows yet. There is more recent commentary here: https://github.com/pex-tool/pex/issues/259#issuecomment-2049980527 but one very broken thing is file locking which Pex relies on in its cache handling. That is currently Posix-only. There are many other issues though, including handling venv .exe console script wrappers and more.

My basic stance is that I need green CI on a platform to support that platform. We don't have CI setup for Windows yet. I put a fair chunk of effort towards this in https://github.com/jsirois/pex/tree/Windows/wip ~2 years ago. I'm picking that up again, but you'll truly know when Pex supports Windows by looking here in the "Operating System" section: https://pypi.org/project/pex/ (or here: https://github.com/pex-tool/pex/blob/1921ba46ca378b537d6917f853b832d32eb80637/pyproject.toml#L38-L40)

akrieger commented 3 months ago

Well, I wrote this because it fixed an issue we were having and I wanted to make sure no one else it hit too. When the project is ready to merge this, it'll be available.

jsirois commented 3 months ago

Thanks. I'll make sure to include this in the initial Windows support release.

Well, I wrote this because it fixed an issue we were having and I wanted to make sure no one else it hit too.

The thing is, no one at all should be using Pex on Windows right now. I can't support it sanely without (green) CI or a Windows-using prospective Pex user providing that green CI baseline.