search

Azure / azure-uamqp-python

AMQP 1.0 client library for Python
MIT License
55 stars 47 forks source link

Wheel support for linux aarch64 [arm64] #191

Open odidev opened 3 years ago

odidev commented 3 years ago

Summary Installing azure-uamqp-python on aarch64 via pip using command "pip3 install azure-uamqp-python" tries to build wheel from source code

Problem description azure-uamqp-python doesn't have wheel for aarch64 on PyPI repository. So, while installing azure-uamqp-python via pip on aarch64, pip builds wheel for same resulting in it takes more time to install azure-uamqp-python. Making wheel available for aarch64 will benefit aarch64 users by minimizing azure-uamqp-python installation time.

Expected Output Pip should be able to download azure-uamqp-python wheel from PyPI repository rather than building it from source code.

@azure-uamqp-python-team, please let me know if I can help you building wheel/uploading to PyPI repository. I am curious to make azure-uamqp-python wheel available for aarch64. It will be a great opportunity for me to work with you.

yunhaoling commented 3 years ago

hey @odidev , apologize for keeping you waiting for such a long time.

If I'm understanding it correctly, aarch64 is a subset of the ARM architecture, so the question here is to provide pre-built wheels for the ARM architecture (ARM32/64). And your suggestion sounds reasonable to me.

I'll bring team's attention to this topic of multi-arch wheel support and let you know how we wanna approach this. cc: @lmazuel @annatisch

odidev commented 3 years ago

@yunhaoling, did you get any suggestion form the team on multi-arch wheel support?

odidev commented 3 years ago

Manylinux2014 ensures that the wheel is built against the AArch64 v8.0 specification. All AArch64 platforms in the v8 series maintain backward compatibility with v8.0. So a binary built for v8.0 can run on any v8 platform. Enabling aarch64 wheel support will benefit arm v8 series users by reducing installation time.

@lmazuel, @annatisch, @yunhaoling can you please let me know if you are interested in releasing aarch64 wheel support.

yunhaoling commented 3 years ago

hey @odidev, we would be happy to add support, but we first need to investigate how to manully build the whls on the the AArch64 platform.

basically to determine whether we need new VM agents with ARM architects, what're the supported python version, do we need new docker images for the build etc.

It would be nice if you could tell us more about the machine spec/os images you use (links, references, downloads etc.). 😊

odidev commented 3 years ago

Hi @yunhaoling , I tried building aarch64 wheels in manylinux2014_aarch64 docker container on my local aarch64 machine by using similar script as " build_many_linux.sh". Attached the script build_many_linux_aarch64.txt for your reference. The wheels are getting generate successfully.

If you don't have aarch64 machine, you can try the steps on travis-ci.com. Please let me know if anything else is required from my side.

odidev commented 3 years ago

@yunhaoling, please let me know if anything is required from my side. Thanks.

odidev commented 3 years ago

@yunhaoling, please let me know if I can help you building/uploading aarch64 wheels. Thanks

yunhaoling commented 3 years ago

hey @odidev , thanks for checking.

Apologize for missing the thread here -- I have been working on some urgent bug fixing issues these days. But the arm64 wheel support is always on my mind, I'll probably have some time to look next week.

Our plan in general on this is to

  1. Build the wheels locally by the information you provided on our dev boxes (thanks again)
  2. To investigate how to integrate the extra build process with our internal devops system

Thanks for your patience and understanding in advance.

odidev commented 3 years ago

@yunhaoling, Could you please let me know if there is any update regarding this?

yunhaoling commented 2 years ago

hey @odidev , sincerely apologize for the late response. I was trying to use qemu to simulate a ARM 64 machine but the progress is slow.

On the other side, instead of supporting multiple platforms by building/compiling wheels, our team endeavors to get a pure python implementation of the amqp library which could help us get rid of this situation. (work is happening on the branch pyproto

I believe the pure python version is the long-term way we want to go and could resolve your issue.

robcowie commented 2 years ago

Hi. Is the decision of the maintainers not to provide pre-compiled wheels for aarch64?

I can certainly understand why if you are aiming to move to a pure python implementation but right now it causes difficulty when installing in potentially contained environments (i.e. a docker image without build tools) on arm hardware (i.e. new Apple gear).

yunhaoling commented 2 years ago

hey @robcowie , we understand that the aarch64 support is wanted and we have put efforts into how it could be supported. We succeeded to manually build uamqp wheel on a M1 Apple Mac without getting too much trouble -- install cmake first and then run pip3 install uamqp --no-binary :all worked.

However, we depend on internal Azure Devops VMs to build binary wheels, previously there was no ARM64 VMs being supported which means we just can't build and ship ARM64 wheels. The good news is that I learn last week that internally the ARM64 support VMs (Windows and Linux) are going through previews, and I would love to dogfood it and see how it goes.

robcowie commented 2 years ago

Thanks for the info @yunhaoling

davidcroda commented 2 years ago

Is this still on the roadmap?

yunhaoling commented 2 years ago

hey @davidcroda , yes, it's still on our track. We're still monitoring the azure devops agent support waiting for its public support.

are you looking for the uamqp library itself or you're using azure-eventhubs/azure-servicebus python sdk on aarch 64(arm64)?

the reason for asking you the question is because we're also working on pure python based amqp library which is being publicly developed in this branch: https://github.com/Azure/azure-uamqp-python/tree/pyproto.

And we've chosen to start the alpha of pure python amqp library as a vendored package within our Azure EventHubs and ServiceBus SDKs while we get that early feedback before we publish it as a standalone package.

You are very welcome to install and try out from the repo if you're seeking for amqp library or the azure-eventhub alpha SDK if you are trying to use azure-eventhub sdk.

We would appreciate any and all feedback!

samgre commented 1 year ago

Any updates?

kashifkhan commented 1 year ago

Hi all, I wanted to provide an update on this issue:

thank you 🙂

sugawarayss commented 3 months ago

@kashifkhan hi, I want to install uamqp via azure-iot-hub on MacOS 14.4.1, but I ran into this problem. How can I solve it? Or are there any alternatives?