Release conda packages

[jeremiahrose]

Problem: We have an ecosystem that includes bunch of interdependent packages, where half of them are released as Pip packages, and the other half are released as Conda packages.

Example: Using the ocp_vscode extension on an Apple Silicon Mac. This extension requires both cadquery-ocp (which is only available as a Conda package) and build123d (which is only available as a pip package). Installing both requires a nasty workaround that prevents projects from using a straightforward environment.yml file, for example.

Proposed solution: Packages should be released for both Pip and Conda.

How hard would it be to add automated Conda releases to this repository? I would be happy to contribute to this if it's a desired feature.

[jdegenstein]

To my knowledge the only missing package (for a major platform) is cadquery-ocp built for MacOS + aarch64 and released to PyPI. This effort is hampered by GH's lack of MacOS + aarch64 runners. I believe these may be available as of the last few months, but my understanding is that CQ has been unable to use them for building OCP for the aforementioned package/arch.

Further discussion here: https://github.com/CadQuery/ocp-build-system/issues/11

We have a special section in the docs (which you may have already seen) for "Apple Silicon Installs" here: https://build123d.readthedocs.io/en/latest/installation.html#special-notes-on-apple-silicon-installs

Even if we do release build123d for conda, there will still be several dependencies only available on PyPI. So the goal of "all conda" or "all PyPI" is currently unattainable for MacOS + aarch64 anyways.

Regarding your point about ocp_vscode requiring build123d, I do not see that in the project dependencies here: https://github.com/bernhard-42/vscode-ocp-cad-viewer/blob/1a1dfb2a7cb3d78ab228a7a80307f6f39aaa897f/setup.py#L7 Am I missing something?

[jeremiahrose]

To my knowledge the only missing package (for a major platform) is cadquery-ocp built for MacOS + aarch64 and released to PyPI. This effort is hampered by GH's lack of MacOS + aarch64 runners. I believe these may be available as of the last few months, but my understanding is that CQ has been unable to use them for building OCP for the aforementioned package/arch.

Further discussion here: CadQuery/ocp-build-system#11

This matches my understanding too - Github does not currently have a free-tier for osx-arm64 runners, so open source projects without funding are unable to access them. And there is no ETA from Github for when these might become available. This makes publishing wheels difficult. However, Conda Forge uses cross compilation in their buildchain so osx-arm64 Conda Packages are achievable now.

We have a special section in the docs (which you may have already seen) for "Apple Silicon Installs" here: https://build123d.readthedocs.io/en/latest/installation.html#special-notes-on-apple-silicon-installs

This is incompatible with installation via environment.yml files and is not a very user friendly solution unfortunately :(.

Even if we do release build123d for conda, there will still be several dependencies only available on PyPI. So the goal of "all conda" or "all PyPI" is currently unattainable for MacOS + aarch64 anyways.

Yes, we would need Conda Packages for the dependencies as well. I think this is achievable - I have raised issues on a couple of them. It should be fairly easy to set up.

Regarding your point about ocp_vscode requiring build123d, I do not see that in the project dependencies here: https://github.com/bernhard-42/vscode-ocp-cad-viewer/blob/1a1dfb2a7cb3d78ab228a7a80307f6f39aaa897f/setup.py#L7 Am I missing something?

Unfortunately it is required but not declared as a dependency. See issue here.

[jdegenstein]

I personally believe that the correct approach here is to find a way to automatically build/release cadquery-ocp for MacOS + aarch64 to PyPI. I did some research and it appears that cibuildwheel has the ability to cross-compile aarch64 (aka arm64) wheels on x86_64. cibuildwheel is also compatible with GH actions, meaning that this should be possible. Since you offered to help, this would be a great help to the entire "OCP community" as it would immediately provide an option for a pure PyPI install method for all major platforms.

Further reading: https://cibuildwheel.readthedocs.io/en/stable/faq/#apple-silicon

https://cibuildwheel.readthedocs.io/en/stable/faq/#how-to-cross-compile

[jeremiahrose]

Agree that getting cadquery-ocp binaries for osx-arm64 is a good way forward.

Unfortunately the ocp-build-system is written in an unusual way - instead of compiling OCP it just downloads the existing Conda packages and repackages them as wheels using a long and compicated setup.py. Which means that repo is not really compatible with cibuildwheel. And it can't do cross-compilation because the setup.py imports OCP while repackaging it, which doesn't work for non-native architectures.

The best solution I could find was to use ocp-build-system to just package the wheels locally on my M1 Mac and upload them. I've submitted a PR to ocp-build-system containing the script for doing that. Since the releases are triggered manually anyway, adding this addition manual step is not really a major burden. Until that PR gets merged I've published the locally built wheels here. They can be added directly as URLs to a requirements.txt etc.