search

piwheels / packages

Issue tracker for piwheels package issues
https://github.com/piwheels/packages/issues
20 stars 5 forks source link

Package issue: kivy does not support BCM/DISPMANX backend `egl_rpi` on piwheels #289

Closed peanball closed 11 months ago

peanball commented 2 years ago

Package name

kivy

Package version

2.0.0

PyPI URL

https://pypi.org/project/Kivy/

piwheels URL

https://www.piwheels.org/project/kivy/

Python version

I am the maintainer

More information

The egl_rpi window backend does not work on raspberry pi < 4 even though it should.

Kivy's setup.py detects additional modules based on availability of files or other indicators in the system. The Raspberry Pi EGL cython modules are only built when the build is run on a Raspberry Pi or with set cross compilation flags.

The cython .so files for egl and bcm are not in the package on piwheels, which are required for the egl_rpi window provider. This is only the case when the compilation platform is not detected as Raspberry Pi (< version 4).

Those files are in the wheel that is on the GihHub Releases page for Kivy (for arm wheels) and on the package retrieved from PyPI!

Please either adjust the build or even better, take the built wheels as provided by Kivy, which work perfectly fine on Raspberry Pi as they are.

Related issue on Kivy (will be closed as it is not a kivy issue!): https://github.com/kivy/kivy/issues/7806

Related PR on Kivy that might help piwheels in forcing the build of cython modules for the egl_rpi mode in Kivy: https://github.com/kivy/kivy/pull/7804

The Raspberry Pi build for ARM on Kivy is working and carried out here: https://github.com/kivy/kivy/blob/master/.github/workflows/rpi_wheels.yml and https://github.com/kivy/kivy/blob/master/.ci/ubuntu_ci.sh#L215

peanball commented 2 years ago

The most likely cause is that the package is built on a Raspberry Pi 4, which does not support the BCM/DISPMANX EGL libraries. The Kivy project has a detection in the setup.py and will refuse to build those libraries on a Raspberry Pi 4.

The PR https://github.com/kivy/kivy/pull/7804 introduces a flag to force a lower RPI version and likely conclude the build with all features intact. This has not been tested yet building the project on a Raspberry Pi 4 while forcing it to think it's running on a Raspberry Pi < 4.

I don't know how the build system / customisability of piwheels and whether this could be set up to make kivy build with specific env variables set when running on piwheels. I'd be happy to help with getting the build to work as needed and verify the outputs.

bennuttall commented 2 years ago

We don't currently have a mechanism for customising builds.

Does this affect all the 2.x releases so far? And will it continue to affect future builds?

We are able to import wheels, so that's an option. We can import separate wheels for armv6 and armv7 but not separate ones for Pi 4. If the wheels we host do not work on Pi 3 then they'll need removing. If Kivy produce their own wheels which are compatible with ALL Pi models then I'll be happy to import them. Can you just confirm which versions are affected and confirm that those wheels are compatible with all Pi models?

matham commented 2 years ago

The current piwheels wheels do work for both Pi < 4 and Pi 4. However, there are extra features that could be enabled for Pi < 4 that the Kivy built wheels include but that the piwheels don't include. So there's no need to remove them or change anything.

Going forward however, it would be nice if we can upload/import the wheels to piwheels. What is the process? We upload them to our release page. Is that where you'd import it from?

We include wheels for both armv6 and armv7, but they are the same wheel, just copied and renamed.

peanball commented 2 years ago

@matham, you also publish them on PyPI, I guess this is where piwheels could fetch them.

@bennuttall, I will go through the versions you have on piwheels to identify the ones missing the .so files I mentioned for a complete list. It is possible this started before 2.0.0.

For context: @matham is one of the maintainers of Kivy

peanball commented 2 years ago

@bennuttall, I went through all versions that built on piwheels.

All of them do not have the necessary egl and bcm Cython .so files.

Anything before kivy 1.10.1 did not build successfully on piwheels. Already with the first successful version 1.10.1, the libraries are missing.

It would be great if you could pull the releases produced by Kivy instead, for all releases. Not sure if anything before 1.10.1 makes sense, as even Kivy hosts docs on their website only starting with this version.

Thanks in advance!

matham commented 2 years ago

@matham, you also publish them on PyPI, I guess this is where piwheels could fetch them.

We don't. We publish the manylinux wheels, not arm wheels.

It would be great if you could pull the releases produced by Kivy instead, for all releases. Not sure if anything before 1.10.1 makes sense, as even Kivy hosts docs on their website only starting with this version.

Kivy only started to make functioning arm wheels from the last release on, I think. So prior releases would not be available. But even for the last release I don't think it's a good idea as that may potentially break people's existing apps if they built it around piwheels compiled kivy. All of sudden pulling in a kivy built wheel for an older release may break something. E.g. pypi has the policy that you can never replace existing release artifacts for this reason!! You can only create new releases or delete old releases, but not replace them, for this reason.

So, I think the best path forward is to keep the existing wheels as is and figure out a way to get kivy built wheels uploaded to piwheels.

peanball commented 2 years ago

All of sudden pulling in a kivy built wheel for an older release may break something.

Yes, that's fair. The piwheels built wheels include the kivy examples as well.

So would this be something for the next release of Kivy, 2.1.0?

matham commented 2 years ago

So would this be something for the next release of Kivy, 2.1.0?

Yes!

peanball commented 2 years ago

@bennuttall, kivy 2.1.0 is now released. How would we go about using kivy produced wheels on piwheels?

bennuttall commented 2 years ago

Thanks - do you have them hosted? If so, post a link here, otherwise email me: ben@bennuttall.com

peanball commented 2 years ago

Hi, yes they are hosted by the Kivy team via GitHub releases: https://github.com/kivy/kivy/releases/tag/2.1.0

Specifically the arm wheels: Kivy-2.1.0-cp37-cp37m-linux_armv6l.whl 5.97 MB Kivy-2.1.0-cp37-cp37m-linux_armv7l.whl 5.97 MB

It's the same file renamed, but runs on arm v6 and v7.

bennuttall commented 2 years ago

Ok great. I can import them. Would you be able to build cp39 wheels too?

peanball commented 2 years ago

@bennuttall, I'm not a maintainer. I can request / push that forward for a next release maybe.

matham commented 2 years ago

I just made https://github.com/kivy/kivy/pull/7852 to see if it'll build for 3.9 easily.

misl6 commented 1 year ago

I guess we can close this issue as WONTFIX (CANTFIX).

DISPMANX is not available on Raspbian Bullseye, but we managed to have support for KMSDRM via our SDL2 window provider, which we now self-build. (We got a confirmation here: https://github.com/kivy/kivy/pull/8223#issuecomment-1529051541).

Kivy's CI/CD jobs and setup.py has been updated accordingly. (See full story: https://github.com/kivy/kivy/pull/8223)

Even if piwheels will not be able (not due to its fault) to "automagically" build the packages for Kivy, the Kivy team will provide cross-compiled wheels to piwheels maintainer @bennuttall (https://github.com/piwheels/packages/issues/356#issuecomment-1529740330) from now on.

peanball commented 11 months ago

@misl6 that works for me. Thanks.