BUG: windows arm64 support

fox4u commented 12 months ago

What pyzmq version?

25.1.1

What libzmq version?

4.3.4

Python version (and how it was installed)

CPython 3.11.4

OS

win11 arm64

What happened?

Bundled libzmq version does not support windows arm64. Support has been added around this commit https://github.com/zeromq/libzmq/commit/e9f86fee61ceeb5658462784d68449ed76ae50ba , and the latest revision on master branch should work well. Since 4.3.4 has been 2+ years old and 4.3.5 is not officially released, would it be possible to add support of fetching a specific revision rather than the release archive?

Code to reproduce bug

No response

Traceback, if applicable

No response

More info

No response

minrk commented 12 months ago

would it be possible to add support of fetching a specific revision rather than the release archive?

Since you're building from source, you can clone the libzmq repo at the desired ref into bundled/zeromq and it ought to work.

fox4u commented 11 months ago

Since you're building from source, you can clone the libzmq repo at the desired ref into bundled/zeromq and it ought to work.

Most of the time pyzmq is installed as a dependency when doing pip install other package. And since there's no prebuilt win arm64 wheel currently in pypi it will be built from source. And for this build it is not possible to clone libzmq in bundled/zeomq. People might just have problem using the other package, like me with jupyter, and never realize that it is a problem in the bundled libzmq here.

minrk commented 11 months ago

Ah, got it. So the request is for installing an unreleased build of libzmq by default so we can support Windows on arm. That's tricky, but since it's I'll see about that, or also if it's a small patch, applying a backported patch to our bundled source (I'm not sure). I also think there's been some work toward 4.2.5, so maybe that release will come before this gets done.

fox4u commented 11 months ago

So the request is for installing an unreleased build of libzmq by default so we can support Windows on arm.

Such unreleased version isn't necessarily to be installed by default. Since we already have the flexibility of ZMQ_PREFIX env var, we might be able to make an extra option, e.g. ZMQ_PREFIX=github:\<rev> to clone from official repo into bundled/zeromq for the build. Besides windows arm64 support, this change may also benefit the pyzmq dev'er. If you think this is viable I will try to submit a PR.

minrk commented 11 months ago

Sure, if you want to give ZMQ_PREFIX=git: (or https:// or something) a try, that would be cool. A generic git clone or git-archive URL. I guess clone URLs don't support refs, so you'd have to define e.g. a pip-style url@ref pattern (I'd probably use exactly pip's, since this is for the Python installation process).

fox4u commented 11 months ago

Proposed code changes in here https://github.com/fox4u/pyzmq/commit/0473c13 . An url[@ref] pattern in ZMQ_PREFIX is supported. For example:

ZMQ_PREFIX=https://github.com/zeromq/libzmq.git
ZMQ_PREFIX=https://github.com/zeromq/libzmq.git@4097855

Tests have been done on win11 arm64 / x64, linux x64, macos arm64. pip_install_win11_arm64.log pip_install_win11_x64.log pip_install_linux_x64.log pip_install_macos_arm64.log

Please comment.

minrk commented 11 months ago

Can you add git@ prefix for a git clone, and make a pure URL expect a tarball, e.g. =git@https://github.com/zeromq/libzmq.git@4097855 or =https://github.com/zeromq/libzmq/archive/4097855.tar.gz? Otherwise, 👍

fox4u commented 11 months ago

Sure, but maybe we can just do a tarball download once the URL has a .tar.gz suffix, and git clone otherwise, without the need of git@ prefix. What do you think?

fox4u commented 11 months ago

On the other hand, archive zip is also supported on github, i.e. https://github.com/zeromq/libzmq/archive/4097855.zip is valid. Shall we just support the .zip archives, instead of the tarball, and the existing release zip fetching code can be simply reused?

minrk commented 11 months ago

without the need of git@ prefix

I was thinking it would be nice to match pip's URL scheme, since this is part of Python package install so folks might already be in a pip mindset, and they use git@.

existing release zip

Yeah, go ahead and re-use zip for that, that makes sense.

fox4u commented 11 months ago

I have made the changes to support git@ prefix and zip archives. https://github.com/fox4u/pyzmq/commit/74027fd

# zip archives
ZMQ_PREFIX=https://github.com/zeromq/libzmq/archive/ecc63d0.zip

# git clone (default branch latest)
ZMQ_PREFIX=git@https://github.com/zeromq/libzmq

# git clone and checkout some ref
ZMQ_PREFIX=git@https://github.com/zeromq/libzmq@ecc63d0

Test log: pip_install_git_clone_and_zip_archive.log

minrk commented 11 months ago

Nice! Open a PR?

zeromq / pyzmq