[REQUEST] More tools in Python

[tsvilans]

Is your feature request related to a problem? Please describe.

Since the current Python module only implements basic manipulation of grids, it would be very useful to have more access to tools such as GridTransfomer, GridSampler, etc.

Describe the solution you'd like

Since templated classes are a bit more tricky to wrap in Boost.Python, an example of how this is done with one of these tools would help implement others.

[mifth]

I assume PyOpenVDB is just an example of using OpenVDB+Python. PyOpenVDB had no love since OpenVDB V3. I can also assume that building and supporting this part is a bit problematic.

I can suggest using Ctypes instead of Boost and create your own Py system. Just my 5 cents.

[Idclip]

I think @mifth's synopsis is generally correct. I know that the maintainers would like to extend the python bindings but I'm not aware of any work planned in the near future.

What has been discussed extensively is the idea of removing boost python in favour of a more lightweight alternative (the main contender being pybind). I think that the idea of such a refactor is somewhat blocking adding new bindings so hopefully this discussion picks up some more traction.

[tsvilans]

Thanks both @Idclip and @mifth ! That gives a much clearer picture.

Indeed, I would support something like pybind, since I have already used it to wrap some OpenVDB functionality successfully in a project, and it is much easier to compile without Boost. I've now tried to switch over to PyOpenVDB so that I'm using the "official" Python bindings, but I see that maybe I should continue with my original route.

If there is a developing discussion among the maintainers about this, I'd be keen to follow it and can hopefully contribute 👍

[Idclip]

If there is a developing discussion among the maintainers about this, I'd be keen to follow it and can hopefully contribute 👍

I did a quick search in our meeting notes with a few results (I definitely feel like we've discussed this more 🤔, though if you instead search for "dependencies" that's probably why):

(do a CTRL-F for pybind in these) https://raw.githubusercontent.com/AcademySoftwareFoundation/openvdb/master/tsc/meetings/2021-02-16.md https://raw.githubusercontent.com/AcademySoftwareFoundation/openvdb/master/tsc/meetings/2020-06-04.md

You can monitor the mailing lists/weekly meetings here https://lists.aswf.io/g/openvdb-dev. Things are a bit busy with the 8.1 release but I will look to raise this once 8.1 is out and things have settled a bit. At the very least a formal decision on replacing boost_python would be ideal.

[tsvilans]

Thanks @Idclip , I haven't yet familiarized myself completely with this group, meetings, etc. Good pointers, and I'll check out the mailing list.

Looking forward to 8.1 and the Python discussion afterwards! My vote would be for a closer look at pybind :)

[Idclip]

@tsvilans, there was some discussion about this last Tuesday, with a motion proposing a move to pybind: https://github.com/AcademySoftwareFoundation/openvdb/pull/1094

Also a more public e-mail chain here: https://lists.aswf.io/g/openvdb-dev/topic/replacing_boost_python/83416891?p=,,,20,0,0,0::recentpostdate%2Fsticky,,,20,2,0,83416891

There will most likely be some more discussion around this so please reach out on the lists should you have any feedback! Anyone is also more than welcome to attend the Zoom sessions (you'll find the invites posted in https://lists.aswf.io/g/openvdb-dev/topics) if you spot any agenda topics posted which look interesting.

[chrisflesher]

Hello, I might be interested in helping out with pybind11 upgrade if anyone can point me in the right direction. Am very interested in getting the GridTransformer stuff to work in python.

[tsvilans]

Great, thanks @Idclip ! Will have a read. I've had a very busy time lately, but hope to be able to take a closer look over the summer.

[diiigle]

if anyone can point me in the right direction

@chrisflesher I left a comment in the meeting notes PR the other day.

I am similarly interested in contributing, but can't promise any commitment. But I am definitely available for advice. (Disclaimer: I haven't converted boost.python projects before, only pybind projects from scratch)

• [ ] I would start by setting up the build infrastructure: integrating pybind11 into CMake, either as a git submodule (unsure about OpenVDB's policy about that, maybe @Idclip would know), or by requiring the user to pip install pybind11.
• [ ] Then trying to swap out the boost headers for pybind headers and trying to get it compiling again with minimal syntax tweaking. If that doesn't lead to quick results I would honestly resort to writing the bindings from scratch with the 'new' syntax.

A bigger design decision at the beginning is where to put the binding code, if it either:

1. lives next to the source code (maybe in a python subdirectory) and similar filename as the C++ code it binds
2. or, if one continues with this out-of-source-style separate binding folder in the root directory.

[Idclip]

Sorry for the delay replying here. I've created a new issue which specifically aims to track the work required for a migration to pybind11: https://github.com/AcademySoftwareFoundation/openvdb/issues/1124. If anyone has any more comments, suggestions or contributions please put the there!