Bindings to scripting languages (e.g. via SWIG)

[kovasb]

Thanks for the amazing project!

It would be great to get more documentation on the xml formats supported by the viewer. There are several different ones used by the examples on the demo page, but they are not documented afaict. Its not very user-friendly to reverse-engineer the formats from the existing examples and the code.

Particularly for new users this could be a great way to explore Ospray functionality, figure out how things work, and assess quality/speed without having to write & compile new code.

My use case is I'm looking for a CPU-based renderer for scientific visualization of a large # of datasets but am not a C guru, and hence the XML formats are a very appealing interface for getting started. Since I'm assessing several approaches, being able to easily try things is important.

If writing documentation is too tedious, even just adding more test cases or examples in xml format that cover the spectrum of functionality would be very helpful.

Thanks!

[jeffamstutz]

Hi,

Yes, you've definitely found quite a soft spot in our releases. Some of the history here: we used to have N different example applications all showcasing subsets of functionality that OSPRay provides. In recent releases (v1.4.x and beyond), we have consolidated these apps into a single viewer, ospExampleViewer. In this consolidation, we've had to maintain some legacy methods of importing data to keep our website demos running: there used to be 4 separate XML formats we could ingest, so we are actively trying to reduce this (it will be 3 in v1.5.0). So some of the problem lies in the fragmentation of our data importers for the example apps, though at least there's one app they import into...

One of our target solutions is to have a single XML format that matches a scene graph implementation (on top of the OSPRay API, will be a part of our upcoming v1.5.0 release). This would give us a way to specify anything that OSPRay can digest: both scene data and rendering parameters/objects (i.e. lights). Once we are happy with this XML format, we will deprecate the old ones and update the website demos to use the new one (and document!). Unfortunately we need to get this release out soon (targeting tomorrow at the moment) to deliver core OSPRay API features, but we are aware of the power a solution like this would hold.

One option you could use for evaluation is to download ParaView and check the "enable OSPRay" box in the left-hand panel. As a production visualization applications, ParaView supports many data importers and has documented file formats you could use to import data. The pre-built ParaView binaries ship with OSPRay "out-of-the-box", though if you choose to build VTK/ParaView, you'll have to explicitly enable OSPRay as a build option.

Regardless, we hear you and hope to continuously improve in this area as OSPRay moves forward!

Cheers, Jeff

[kovasb]

Hi,

Glad to hear there is active work on the underlying issues!

I've spent some time looking into vtk & paraview, and the bottom line is I couldn't figure out how to get them to work in my environment of choice (colab.research.google.com). I was able to get VTK to run there but looks like it didn't ship with Ospray enabled for anything besides volume rendering. In general its not clear VTK & Paraview will support the different geometries & materials that Ospray does, but it might just be my lack of understanding.

Hence the appeal of using Ospray directly since its API seems pretty clean and apart from the xml format, well documented.

My ideal workflow is using Jupyter (typically in colab) to generate voxel renders, a la magicavoxel, and once I find satisfactory parameters, to run it in batch to generate 1000s of images.

One question you might be able to help with: what is the right Ospray APIs to generate magicavoxel-like renderings? So far I've generated a naive mesh for 1000s of cubes, but was wondering if I should be using one of the volume formats, or do isosurface extraction. My initial experiments worked but performance is not what I've seen in some of the examples, and it seems to crash the viewer at times (my guess is because my mesh is not very optimal, with duplicate faces etc).

[jeffamstutz]

Hmmm, interesting setup. I don't know anything about colab.research.google.com, but it looks very Python heavy (as is Jupyter?).

We contribute regularly to VTK to make sure that OSPRay's capabilities are used to their fullest extent. We've been a main rendering backend in VTK (both surfaces and volumes) since 7.1. VTK supports OSPRay triangle meshes, spheres, and cylinders. However, if the VTK route won't work in the end, that's OK too. :)

OSPRay is just a rendering library (conceptually the same role as OpenGL or Direct X), so while we do ship an example application for users to explore the OSPRay API and its features, it isn't really the "product" we are trying to ship (hence the reason our XML fragmentation issues both exist and are not the first things we go after to fix).

That said, you probably have a few options, but it looks like you'd need to write some C++ with a custom application that could take data from your Jupyter environment, setup a camera/lights/materials/etc, and rendering images which could be displayed back in your environment. One thing that would make this much easier would be a layer which binds OSPRay API calls to a scripting environment (i.e. Python). While I think that is a bit outside the scope of this particular source repo, I do think that could be a valuable library project to build on top of OSPRay. While it sounds useful, we haven't worked on it because we haven't had a user explicitly ask for it....would Python bindings to the OSPRay API make sense, or are you after something else?

As for your other issue with performance, less primitives should indeed be faster! After looking at magicavoxel (another cool project I hadn't heard of!), I think a custom mesh/volume might make sense, though if it's all opaque geometry then you could just create a mesh from the resulting voxels. It would really depend on the exact visualization you are going for. The process for adding new primitives is to either a) use the SDK headers in the OSPRay install to write your own custom extensions (i.e. new geometry/volume types), or b) wait for us to implement it (which could be a while).

[kovasb]

Thanks for the insight. This makes sense. Yes colab is a Jupyter-based thing.

A Python API into OSPray would be very cool and seems like it would be easy to write against the C api. Simply using CFFI seems doable for starters. Could be useful for prototyping and demos.

Though its not the primary use case, there's some unique advantages of using OSPray as a more user-facing renderer:

AFAICT the number of fast, open-source, well-maintained, portable, programmatic-API friendly renderers is very small. VTK is pretty impressive tech but also hard to swallow as a dependency for many projects.

Currently, if you want to programmatically generate geometry and render to high quality, its a big chore. If you want this to be portable, you are in for a headache. You have to tradeoff against one of the dimensions (portability, cleanliness of API, speed, etc). If you wanted to, for example, integrate a nice renderer into Processing or Mathematica, whats out there?

This is probably more of an argument for language-specific bindings (python & JVM would be my personal choices) than for XML formats, but its getting at the same need.

Anyway thanks for taking this input!

[jeffamstutz]

We discussed this in our weekly team meeting yesterday, and the leading candidate is to employ SWIG to give us several targets for OSPRay API bindings (which includes Python!). We can't commit to a time-table right now, but it sounds like something we'd like to get to eventually.

[BruceCherniak]

Here's a start that Tanner @ UTenn did https://github.com/player1537/PyOSPRay/

[paulmelis]

I missed the above recent update to this issue with the link to PyOSPRay and had started noodling with pybind11 myself. So there's an alternative set of (Python-only) bindings at https://github.com/surfsara-visualization/ospy. The bindings are not complete, but I'm amazed at how easy pybind11 makes the whole process, given that that repo contains pretty much a day's work.

Anyways, just wanted to share.