CUDA support

[jeremyselan]

Add CUDA support, per http://groups.google.com/group/ocio-dev/browse_thread/thread/73935241bcd5ff1a

[jeremyselan]

@nweston, Gets much further now, but having this error: https://gist.github.com/2402166

[lgritz]

Adding myself. Is there a way to request to follow an issue without adding a useless comment?

(And don't even get me started about how stupid it is that GitHub doesn't let non-committers subscribe to get all issue and pull request emails.)

[jeremyselan]

@lgritz, Not anyway that I know of for a non-admin to get cc'd or added to a topic. I'll look into it. I feel there should be a way...

[nweston]

Oops, I hadn't tried building the tests (I guess they're off by default). Should be fixed now.

[jeremyselan]

I can confirm the builds and works now. Sweet!

In terms of an architectural review of the CUDA branch, I'm heading out of town for a few days but will study this in transit. So let's pick it up next week, ok?

[malcolmhumphreys]

This is looking really cool, great job.

From an ABI point of view it would be nice to have cuda mock / dummy signatures where possible rather than the ABI changing when you compile with cuda or not.

It would be great to have a single ocioconvert binary which supports both cpu and cuda paths.

It feels like it would be nice to split cuda support out into it's own 'libOpenColorIO-Cuda.x.x.x.so' lib. Maybe when a Config is setup the symbols (using dlopen(), dlsym() etc) could be resolved while also checking if the device is capable of what we need it todo. (This helps to resolve some linux distro packaging potential issues, on debian you could have a non-free deb for the cuda support)

We could also add a Processor::supports('device enum' CG | GLSL_1_3 | CUDA_4_1 ) which could be used to detect if a processor can run, so the host app can choose to fallback to the cpu path if desired.

[nweston]

From an ABI point of view it would be nice to have cuda mock / dummy signatures where possible rather than the ABI changing when you compile with cuda or not.

Makes sense. I initially guarded everything in OCIO_BUILD_CUDA in case any of the new definitions depending on CUDA headers. But that didn't happen much, so it should be easy to remove the OCIO_BUILD_CUDA guards at least from OpenColorIO.h

It would be great to have a single ocioconvert binary which supports both cpu and cuda paths.

Agreed.

It feels like it would be nice to split cuda support out into it's own 'libOpenColorIO-Cuda.x.x.x.so' lib.

Makes sense. I'll probably leave that kind of thing until the end when I'll have a better idea of how to design the interface between the CUDA and non-CUDA bits.

[jeremyselan]

Sorry I havent gotten around to reviewing this further. Will do so ASAP! :)

In terms splitting into CUDA/non-CUDA, that would be a wonderful solution to linking/build issues, but I seems a bit difficult. The hard part is that CUDA will need to inter-operate with the core OpenColorIO library in a very 'private' manner. (I.e., using OCIO internal interfaces / classes that should not be part of public APIs). Can this be done? And then we'll need to be extra careful with the companion library versioning, as the system install will only support matched minor versions of the two. (I.e., we should not have to guarantee that the internal private API is stable in minor releases).

Looking at /usr/lib on my linux box, I see

libnvidia-compiler.so.280.13 libnvidia-glcore.so.280.13 libnvidia-ml.so.280.13 libnvidia-tls.so.280.13

I was hoping that these libraries had link dependencies on each other (as an example of prior art on 'private' APIs), but they do not.

Does anyone know of similar examples? If we went the split library route, would we still be able to get into the major distros?

[nweston]

I took a closer look at the code so far, and I think the interface problem may not be so bad. The CUDA library would have to export: ApplyCudaOps() class CudaOp makeCudaExponentOp() (and the other CudaOp factory functions when they're implemented)

The factory functions will more or less take the same arguments as the corresponding Op constructors. Looking at those, I see only primitive types, or structs and enums from OpenColorTypes.h. And the CUDA implementation so far doesn't call any host code from the main library.

So the CUDA library would have to re-use some code (mostly headers) from the main library, but I don't think we'd have to change the exported symbols. It's hard to be certain until I've implemented the other ops, but I'm optimistic about that.

[michdolan]

Pinging this issue to resurrect it slightly, since CUDA support was discussed in this week's TSC meeting.

[lgritz]

I'd like to raise this old issue from the dead because it's going to be a crisis fairly soon.

I know for OSL, and really any renderer that uses OCIO and is currently scrambling to re-architect a Cuda+OptiX implementation (that is to say, EVERY top film renderer), we are currently faced with three equally unpalatable possibilities: (a) hope that OCIO implements a Cuda-callable interface for most of its major functionality in the reasonably short term, (b) re-implement a good chunk of it in either OSL or OIIO or separately so that we can use it in these renderers instead of OCIO for our GPU code paths, (c) restrict at least the GPU implementation of OSL to only a few simple color spaces (linear <-> sRGB and other obvious things, but definitely not any arbitrary spaces found at runtime in the OCIO config).

As interactive viewports (currently OpenGL or similar) become over time replaced by OptiX or other Cuda-based renderers operating in real time, the current GPU OCIO implementation will not suffice for them and what they really need is to just have a Cuda-based C++ API.

I know the OCIO community has let this lie for a long time in the push to get to OCIO v2 (yay!), but hopefully early in the post-v2 era, this will be perhaps the next major initiative.