[REQUEST] User function support for OpenVDB AX

Is your feature request related to a problem? Please describe.

Currently, it is not possible to define and call user functions from OpenVDB AX programs, where by 'user function' I mean some native function implemented in e.g. C/C++, the address and signature of which would have to be bound to the AX program when it is compiled. A simple example would be to add a new math operator that would be difficult or inefficient to express directly in the AX language, for example some operator that requires accessing a lookup table.

The OpenVDB AX language specification states that "User function declarations are not currently supported, but exist on the AX Roadmap as a near future feature." but without any current status or ETA.

Describe the solution you'd like

Similar to how custom data can be bound to the program using CustomData/TypedMetaData which can then be accessed using the $ operator, there would be a way to bind a function using a combination of its function pointer, function name, and the types of its arguments and return type. The bound function can then be called from the AX program as if it were one of the built-in functions (edit: or, using some special syntax of course, for example something like f@density = f$my_function(f@P))

This is assuming standard C ABI calling conventions, and (at most) support for native AX data types, but even a subset that only includes simple scalars and vectors would already be useful. Thread-safety and memory access/lifetime of any data used by the function can be explicitly left out of scope (ie: 'user code' is entirely responsible to ensure memory & concurrency safety, AX should not need to add any kind of locking or allocation/deallocation).

I expect AX programs that would rely on this functionality can not be compiled to a static/standalone executable anymore, as binding the function code by its address has to be done at run-time, and the resulting binary would have to include all code dependencies of the user function, which I assume is not something the AX compiler can (wants, should want to) do. If support for user functions would be added, this can be an explicit limitation of the feature ('user functions can only be used in AX programs that are compiled inline/on-line, within the scope of a host program that uses OpenVDB/AX')

Describe alternatives you've considered

The most obvious alternative to user functions in AX programs, would be to not use AX programs at all but instead iterate VDB grids directly using the regular OpenVDB C++ API's and apply per-voxel/point logic. This is not really a viable alternative for our use case though, as this would imply any required per-voxel/per-point logic that is not covered by some standard OpenVDB API, needs to be defined at compile-time and linked into the executable. The application we are working on allows user-scripted operations that may require exotic per-voxel operations with highly specific input/data requirements, and trying to cover that by building our own set of primitives for per-voxel operations is not realistic.

Another alternative we've considered is to try to bend the current AX language to our will by passing in things like lookup tables as custom data and accessing them directly from the AX main program, but since AX does not support array-types in any form, this would involve binding separate custom data values for every table element, which is most likely not going to work for a lookup table that could be several KB's of data.

Additional context

The context of this feature is an application that uses an OpenVDB-based backend for modeling volumetric geometries that are highly-domain specific and require operations on the volume data that may only make sense for a single use-case. To accommodate this the application is user-scriptable (using Lua), with user scripts calling into an intermediate API layer that abstracts all OpenVDB details. This works mostly fine for per-volume operations, but it does not work for per-voxel/per-point operations, since calling out to Lua for every voxel/point in a grid to perform custom processing is extremely inefficient. Instead, our Lua scripting engine allows creating 'kernels' defined as OpenVDB AX programs, which can then be applied to the volume. This allows (to some extent) user-defined per-voxel/point operations.

A more concrete example use case that is currently not possible because of the lack of user functions in AX could be a user script that wants to do the following: given a VDB point data grid and a 2D intensity map (bitmap) + associated XY world-coordinate space, assign to each point in the VDB grid a new attribute, the value of which is sampled (bilinear) from the intensity map using the point XY coordinates. To take this idea a little further, instead of 2D intensity map, the attribute values could also be sampled from some other dense VDB grid using more advanced OpenVDB stencils.

This kind of functionality can easily be written using the normal OpenVDB C++ API's, but not by means of an AX program.

Hey @w0utert, thanks very much for the detailed breakdown. If I've understood you correctly, I believe that AX does provide a solution for this. Unfortunately the documentation is (still) lacking in this area so I apologise for this. Specifically to this point:

The OpenVDB AX language specification states that "User function declarations are not currently supported, but exist on the AX Roadmap as a near future feature." but without any current status or ETA.

This refers to user function declaration within the AX grammar/language itself, not support for external bindings.

A similar question was posted here (if you have access to the VDB forums) https://groups.google.com/g/openvdb-forum/c/6MXOzczpx2E but essentially what you want to do is insert a callback which creates a new ax::FunctionGroup into a new ax::FunctionRegistry and apply that to your instance of ax::Compiler.

I've push up a branch here (https://github.com/Idclip/openvdb/tree/bind_user_function_example - https://github.com/AcademySoftwareFoundation/openvdb/commit/752d4b8d4d62a8acd84546c48d2d4d5fe50f117a) which modifies the vdb_ax binary to demonstrate. This branch, when compiled and run with the following command, will add a custom function sum_print which returns the sum of the first and second argument and also prints the 3rd string argument 'test':

vdb_ax cube.vdb -s '@ls_cube += sum_print(1,2,"test");'  -v

There's still a lot of room for improvement in this area but I'm hoping this provides you with a good starting point. Do let me know if this isn't what you're after or is insufficient in any way

Hi @Idclip, thanks a lot for the quick reply and in particular for the branch that demonstrates use of the FunctionBuilder, this is definitely helpful!

I will explore in more detail and do some experimentation on my side, but it seems my use case is largely covered by existing API's, so maybe this ticket should be changed to a documentation improvement task ;-)

Just to be sure, I think this will only allow me to bind custom functions and call them using standard AX types, and only with either literals or variables that were already in the scope of the AX program? In other words: I cannot use this to somehow also bind some opaque pointer value ('user data') to the AX program that can then be used as an argument to my custom function call?

It would be nice if I could, say, bind my_texture as an opaque value, and then do something like this: float@sample = my_texture_sampler(my_texture, vec3f@p) or something like that. I can sort of emulate this by introducing some kind of 'binding slot' concept to my own higher-level API, and write something like my_texture_sample(0, vec3f@p) where 0 is the binding slot. Of course being able to reference user data by name would be much nicer/clearer.

In the end my use case is for AX to act more or less like a fragment shader in a graphics API, but I can see why AX is not 1-to-1 comparable to that ;-)

You can access your own custom data through the the custom data https://github.com/AcademySoftwareFoundation/openvdb/blob/master/openvdb_ax/openvdb_ax/compiler/CustomData.h. An example of how you might add your own bound types can be found in the Houdini AX SOP https://github.com/AcademySoftwareFoundation/openvdb/blob/master/openvdb_houdini/openvdb_houdini/AXUtils.h. The ax function implementation is in hax_chramp and the RampDataCache offers a way of wrapping custom data in a way to be accessed by AX. I believe this should let you do what you need.

Hi @richhones what I would like to do is to bind a native function I can call from the AX kernel, so not just data.

I may need to further investigate the capabilities of passing metadata using CustomData as well though, because as far as see the AX language also does not support array types for custom data. So it seems I cannot easily write a kernel over pointdata, which uses a lookup table passed as CustomData (please correct me if I'm wrong)?

AcademySoftwareFoundation / openvdb