search

numpy / numpy

The fundamental package for scientific computing with Python.
https://numpy.org
Other
28.11k stars 10.14k forks source link

Add Android Support #27698

Open rbebb opened 2 weeks ago

rbebb commented 2 weeks ago

Proposed new feature or change:

Now that Android is a supported platform in Python 3.13, it would be great to have numpy support given that it’s the foundation of many other Python libraries. Tools that let you run Python on Android (like Chaquopy) currently have to build packages like numpy manually and create custom patches: https://github.com/chaquo/chaquopy/tree/master/server/pypi/packages/numpy

mattip commented 2 weeks ago

I had to dig around a bit for context: it seems Chaquopy is the system used to build some python libraries with c-extensions for beeware, which is the goto authority on packaging python for ios and android. As far as I can tell, those platforms cannot install wheels and are using a different packaging system for NumPy.

What would android support look like? What do you call "numpy support"? Is there a way to emulate android on github CI so we can test the eventual build?

It seems the patch you pointed to still uses setuptools, so the Chaquopy build would need to move to meson in order for us to merge the patch. It would be good to add such a build to CI, maybe as an additional job on the wheel build, to make sure we don't break something.

rbebb commented 2 weeks ago

As for what support would look like, I believe a wheel with the native components built for Android is all that would be required.

With regard to using an emulator in GitHub CI, I believe that is possible. I haven't used an emulator in GitHub CI though, and I'm not sure what a test suite would look like.

@mhsmith, I noticed that Chaquopy hasn't added direct support for Meson yet in Chaquopy's build-wheel tool (per this open issue), but do you have more insight on what that would take? Although, perhaps it's possible to build a wheel using Meson directly without using Chaquopy's build-wheel tool? That being said, I don't have experience with using Meson to build Python wheels for Android.

rgommers commented 2 weeks ago

It would be good to add such a build to CI, maybe as an additional job on the wheel build, to make sure we don't break something.

+1 a CI job would be fine if that seems helpful. It'd have a similar status as for Pyodide.

mhsmith commented 5 days ago

The Chaquopy build tool does produce wheels – we currently host them here, and they can be built into Android apps using the Chaquopy Gradle plugin, either directly or via Briefcase. The equivalent for iOS is this tool and this repository.

Now that Android and iOS are officially supported in Python itself, we in the BeeWare project are working on adding support for these platforms to general-purpose packaging tools such as cibuildwheel. The plan is that our existing mobile-specific packaging tools and repositories will eventually become obsolete, and projects will be able to build Android and iOS wheels with their existing workflows, and publish them on PyPI along with all the other platforms.

Testing on Android and iOS emulators is possible on GitHub Actions, and we're already doing it in the Toga project. To see what this looks like, open one of our recent CI runs and look at the "testbed" jobs.

@freakboy3742 is leading the work on this. We're not yet ready to recommend it for production use, but we'll post a comment here once that changes.

freakboy3742 commented 5 days ago

A quick +1 to everything @mhsmith said here.

To set expectations - the focus of the work I'm doing is to get the base tooling of the Python ecosystem into a state where iOS and Android are "just another platform".

The patch for iOS support in pip has already landed in 24.3; iOS support in cibuildwheel is my current project. I've got an initial patch working for simple projects (like lru-dict) that are "pure" binary extension modules with no dependency complications; I'm currently working on getting Pillow building with the same tooling, as it's an example of a package that has complex third-party binary dependencies.

I don't want to push PRs to cibuildwheel for review until I've got a reasonable degree of confidence that they're "fit for purpose"; the work so far has already highlighted some changes that need to be merged upstream to make that work possible (python-pillow/Pillow#8497, pypa/cibuildwheel#2062 and python/cpython#126169 are three examples; I'm expecting to need some more).

Numpy and cryptography are two high-priority packages I'm intending to use as additional source of proof of any cibuildwheel patch (as they're both high profile packages with complex build requirements - Numpy with it's Meson build system, and cryptography with Rust tooling).

Android support is independent of iOS, but there will be a lot of common patterns between the two, so once iOS support is bedded in, the path to adding Android support should be a lot more clear.

So - this will be a long process... but I'm working on it. If anyone wants to help, let me know - I'd be grateful for the assist.