search

ecmwf / ecflow

ECMWF's workflow manager
Apache License 2.0
39 stars 15 forks source link

[question] Any plans to publish the ecflow Python module to PyPI? #24

Open kinow opened 1 year ago

kinow commented 1 year ago

Hi,

I noticed that PyFlow has a dependency on ecflow, but instead of having the dependency listed in setup.py, it's using an environment.yml and Conda.

Would it be possible to publish the ecflow module to PyPI? This way it would be possible for contributors to use just pip to install everything (or just conda, without mixing both unnecessarily).

Thanks. -Bruno

iainrussell commented 1 year ago

Hi @kinow, ecFlow's Python bindings are not a separate Python package, but they are generated by boost-python from the C++ code. Therefore they are built specifically for different versions of Python and I'm not sure if it's possible to have them as a PyPi package. This is a situation we may want to change in the future, but for now this is how it is at the moment, so PyFlow is doing the right thing - it needs ecFlow from conda. Cheers, Iain

kinow commented 1 year ago

Hi @iainrussell ,

I don't know boost or boost-python well enough. But I think it should be possible to build it for multiple platforms.

I found a post that describes what a process that includes several steps that may be similar to what is used to prepare the Conda package, I think: https://thomastrapp.com/posts/building-a-pypi-package-for-a-modern-cpp-project//

Then, for building it against multiple platforms, maybe cibuildwheel could be used. It is used to build PyZMQ, that publishes the platform specific wheels for their Python binding for zeromq: https://pypi.org/project/pyzmq/#files

PyZMQ also uses Circle CI on a free tier, I think (quite sure they have been using it for a long time. CircleCI offers different platforms to build the packages before uploading to PyPI ^1). So technically I believe it's doable, but I understand if there is no interest and, or, bandwidth to try that.

Cheers, Bruno

kinow commented 1 year ago

Forgot to mention that I'd be happy to help with review/test/docs/etc :+1: