Compiled C++ Wheels for PyPi - for CMakeCatchTemplate #41

Closed thompson318 closed 4 years ago

In GitLab by @MattClarkson on Jan 3, 2019, 15:54

I know Steve was battling with some of this, so, until he's back, and we can have an update, here's some thoughts.

Steve was looking at ndicapi and how binary wheels might be produced, and he got the tracking to work, and I can't remember how far he got with pip-installable wheels.
OpenCV has this project: https://github.com/skvark/opencv-python which under the hood using https://github.com/matthew-brett/multibuild/
The reason for such a thing, is that on Mac, its difficult to get a VM setup to build against the right version of Python. You also need to build on the oldest available Mac so its backward compatible. On Linux, you need to try and support numerous distros, so you end up having to use the oldest possible distro, and be able to set it up with the correct python environment. So, thankfully, manylinux has been provided as a docker image for exactly this purpose.
So, if CMakeCatchTemplate is meant to be a template for C++ SNAPPY projects, then is it best to put something like Matt Brett's multibuild (which uses docker on travis) into the CMakeCatchTemplate project so that the end user only maintains one repo? Or is it best to always keep the wheel-building part separate? The downside of the latter option comes if you are using Boost.Python. In your local test environment, you could have Python and Numpy headers installed, and your project compiles nicely on your local machine, but then you wont be able to test on enough versions of compiler and python without using something like multibuild? So a C++ only developer wont want the extra hassle of python bits. But for any given project, if you make the C++ library and Python-Wheel bits into separate repos it may get very confusing.

@ThomasDowrick @MianAhmad @StephenThompson - im not sure what to do here.

In GitLab by @ThomasDowrick on Jan 3, 2019, 17:51

Looks like this is Steve's most recent update on wheels: https://weisslab.cs.ucl.ac.uk/WEISS/admin/issues/26#note_3075

I'll have a more in depth think about the rest of this tomorrow.

In GitLab by @MattClarkson on Jan 3, 2019, 18:31

changed title from Compiled C++ Wheels for PyPi{-.-} to Compiled C++ Wheels for PyPi{+ - for CMakeCatchTemplate+}

In GitLab by @MattClarkson on Jan 3, 2019, 18:31

Thanks Tom. @Steve - looks good.

In GitLab by @StephenThompson on Jan 8, 2019, 09:23

I've got manylinux working via docker on travis and Windows with appveyor. I haven't started battling with mac yet.

In GitLab by @MattClarkson on Jan 8, 2019, 09:24

Might be worth looking at: https://github.com/matthew-brett/multibuild

which is a Multibuild for Linux and Mac, that for the Linux part uses manylinux.

In GitLab by @MattClarkson on Jan 15, 2019, 08:59

FYI - On CMakeCatchTemplate Ive got Mac and Windows builds, but for some reason, docker wont start. (branch 44).

In GitLab by @StephenThompson on Jan 15, 2019, 09:19

I've got binary wheels for ndicapi onto pypi now, via https://github.com/thompson318/ndicapi-wheels for linux and mac and https://github.com/thompson318/ndicapi for Windows.

In GitLab by @MattClarkson on Jan 15, 2019, 09:20

cool. Would you mind digging out a Linux build log to see if I can spot any differences. I've raised a support request with travis-ci.com, but Im lost.

In GitLab by @StephenThompson on Jan 15, 2019, 09:22

Can you access this ...

https://travis-ci.com/thompson318/ndicapi-wheels/builds/96901737

In GitLab by @MattClarkson on Jan 15, 2019, 09:24

Yes. Can you access:

https://travis-ci.com/MattClarkson/CMakeCatchTemplate

Im jealous of your green dashboard!

In GitLab by @StephenThompson on Jan 15, 2019, 09:30

At the top of .travis.yml, try changing the python version from 3.5 to 3.6. 3.5 did not work for me either. As I understand it this is only the python version used for building etc, it is not the version that the wheel gets built for, which is set further down.

In GitLab by @MattClarkson on Jan 15, 2019, 09:35

Thanks. Im off to RSG now.

In GitLab by @MattClarkson on Jan 28, 2019, 08:18

Closing this, as the basic functionality that I had intended is implemented.

What we need now is a good example to actually use the CMakeCatchTemplate, and get our second (after ndicapi) wheel up and running.

In GitLab by @MattClarkson on Jan 28, 2019, 08:18

closed

SciKit-Surgery / scikit-surgery

Compiled C++ Wheels for PyPi - for CMakeCatchTemplate #41