search

myselfhimself / gmic-py

Python binding for the G'MIC Image Processing Framework
https://gmic.eu/
Other
30 stars 4 forks source link

cpython: add support for windows #5

Open myselfhimself opened 4 years ago

myselfhimself commented 4 years ago
myselfhimself commented 4 years ago

This is deprioritized in favor of making first a fully fledged gmic-py v1 for Linux (and Mac OS)

myselfhimself commented 4 years ago

Since we want to bring G'MIC into Blender thanks to this gmic-py binding, looking at Blender's Open Data shows that 2/3 of Blender users are coming from Windows... So Windows support will be needed pretty soon :)

image Per: https://opendata.blender.org/

myselfhimself commented 4 years ago

Consider using cibuildwheel with github action support. See related issue having good progress: https://github.com/joerick/cibuildwheel/issues/159

paulhart2 commented 4 years ago

Attempting to build for Win64 platform. Major errors, not sure how to get going. Thanks for the efforts to get GMIC Python to work in Blender. I will build for GraphicAll.org when it works.

gmic-install_errors.txt

myselfhimself commented 4 years ago

Hello @paulhart2 Thank you very much for the errors reporting. I see how to correct them but will come back to you once I have tried compilation on the Github Actions CI platform which probably has MSVC like you, I will ping you back if I struggle or succeed. Note that gmic-py is indeed intended for any Python 3.5+ environment, but if imported as is within Blender3d, developers using it will need higher skills in order to grab Blender buffers and pipe them into G'MIC or will have to work files only. The latest binary builds of gmic-py are on pypi.org here and I will care that this pypi repository always remains up to the latest version of the underlying G'MIC library as long as I am contracted on this project.

What you may want for Blender3d is maybe the G'MIC Blender plugin, which is to expose more than 500 compositing or texture nodes, but for now that plugin has only Linux support and does not even create 1 node.

I wanted to finish a work on https://github.com/douglaskastle/blender-addon-tester first (very simple MacOSX support). This is a matter of 1/2 day I hope.

Could you tell me what is GraphicAll.org for? Do you build just Blender 3d or a variety of open-source graphics software? How would you see g'mic as a python binding or Blender plugin in it? Thanks

myselfhimself commented 4 years ago

WIP Windows build job on Github Action: https://github.com/myselfhimself/gmic-py/runs/526438299?check_suite_focus=true

myselfhimself commented 4 years ago

@paulhart2 is MSVC compilation a must for you? gmic-py would compile best on Windows using MSYS2 instead from what I have heard. Though I do not want to prevent so many Windows users from self-compiling gmic-py (I intend to release just source and as to Windows, gmic-py Windows binary pre-compiled packages)

myselfhimself commented 4 years ago

Based on this 2017 post on compiling G'MIC on Windows using either MSYS2+Mingw or MSVC, given that the latter MSVC option would probably force to disable OpenMP, I prefer not to even try using MSVC compilation... So Windows Python developers wanting to compile their own binding will have to follow compiling instructions or get inspiration from my upcoming compilation scripts.

I will let you know once I manage to compile gmic-py for Windows, hopefully this week.

jfolz commented 4 years ago

Python for Windows is compiled with a specific MSVC version (see Python wiki). The same MSVC version must be used to compile extensions. I'm not certain whether this applies to dynamically loaded libraries that are used by the extension.

myselfhimself commented 4 years ago

Thank you @jfolz , MingW is also possible up to Python 3.4 according to that same Python wiki.... We could hope that it actually does work for Python versions not just equal to but greater than 3.4.

Possible MSVC versions that I could use (if OpenMP eventually would want to work on it) on Github Actions (I do not to do the windows/packaging locally at all) are listed here for Windows Server 2016, 2019.

This will really be my first compiling a complex project for Windows in the "blind" of a remote CI toolchain. Keep suggesting ideas or try PRs (Github Actions do run on PRs as well) etc..

jfolz commented 4 years ago

Some alternative compilers exist like MinGW, but incompatibilities may occur with a CPython official distribution that is built with Microsoft Visual C++.

Essentially: there be dragons. Segfaults that suddenly occur in 3rd-party libraries, and thus almost impossible to debug.

myselfhimself commented 4 years ago

It has been many years already that gmic compiles and runs well with MSYS2/MINGW, Gimp for Windows also follows that path... Let us experiment, I have a pytest suite that runs for all OSes Windows users such as paulhart2 will be able to test for sure.

myselfhimself commented 4 years ago

Out of curiosity here is a github action to set up msys2 Most important for me is that final final library or wheel needs no extra depencencies apart from what's bundled.

myselfhimself commented 4 years ago

Windows tool similar to macos delocate and linux auditwheel repair, for including DLLs: https://discuss.python.org/t/need-auditwheel-like-utility-for-other-platforms/2028/6

myselfhimself commented 4 years ago

Related dependencies checker for windows: https://lucasg.github.io/2018/04/29/Dependencies-command-line/

myselfhimself commented 4 years ago

Focusing on this for today on until its finished. I will ensure that the virtual machine for building and that for testing are different, namely so that the testing environment does not have any .dll to cheat.

myselfhimself commented 4 years ago

No, focusing on numpy for linux first. Sorry :)

myselfhimself commented 4 years ago

https://github.com/d0m3nik/pycimg/blob/master/.travis.yml shows a windows build using CI Build wheel elegantly, for a similar project (CImg is the backbone of GMic)

myselfhimself commented 3 years ago

@paulhart2 welcome aboard for attempting to compile gmic-py on Windows / MSYS2 by hand firsthand I can then help transpose our efforts into config-as-a-code Github Workflow files containing build and release jobs for every new gmic-py version

myselfhimself commented 3 years ago

Good news, MSYS2 / pacman is already on the Windows images for Github Actions: https://github.com/actions/virtual-environments/blob/main/images/win/Windows2019-Readme.md

myselfhimself commented 3 years ago

Related to https://github.com/msys2/MINGW-packages/issues/7233 OpenMP lib without pkg-config .pc "manifest"

myselfhimself commented 3 years ago

Struggling on the dependent .dll URLs inclusion in wheels..

myselfhimself commented 3 years ago

Ideas for a solution to bundle DLLs: https://vinayak.io/2020/10/23/day-53-a-tool-to-list-pe-file-dependencies/ https://stackoverflow.com/questions/35112511/pip-setup-py-bdist-wheel-no-longer-builds-forced-non-pure-wheels/36886459#36886459