search

NVIDIAGameWorks / FleX

Other
650 stars 100 forks source link

Cuda buffer memory allocation control #73

Open warrickb opened 5 years ago

warrickb commented 5 years ago

Previously in Flex we had explicit control of memory allocation in Cuda so we could serve it from our explicitly managed Cuda memory pools. But now with the advent of NvFlexAllocBuffer() it seems we can't unfortunately.

We'd really like to be able to provide memory from our own Cuda memory pools for Flex to use as we need that kind of fine grained device memory control (we have a lot going on GPU side). Maybe something like a: NvFlexBuffer* buffer = NvFlexRegisterCudaAllocBuffer(ourCudaDeviceMemoryPointer, size)? as I suppose overloads of functions to explicitly take cuda device memory pointers complicates the mapping/sync API stuff.

warrickb commented 5 years ago

Oh hang on I see:

NvFlexUnregisterOGLBuffer() & NvFlexRegisterD3DBuffer() but no NvFlexRegisterCudaBuffer(). Does that mean I can just cast a cuda memory pointer to a (NvFlexBuffer*) ?

warrickb commented 5 years ago

If I can cast Cuda buffers to NvFlexBuffer what does that mean regards syncing with Cuda kernels that interop with those buffers? As I see NvFlexMap() explicitly says: '@param[in] buffer A buffer allocated with NvFlexAllocBuffer()' which makes it sound dubious if a cast is safe in that case?

warrickb commented 5 years ago

And.. what also makes me dubious whether that kind of cast would work is NvFlexGetParticles() seems like it can somehow determine between a host and device pointer, which implies that NvFlexBuffer may not be just a cuda memory pointer and in fact be a CPU side object?

Unless cudaPointerGetAttributes() is being used to check every NvFlexBuffer passed to the API? But that only seems to work in UVA so I doubt it.

So currently it seems there is no way to efficiently provide preallocated Cuda buffers? Unless maybe I hack around by registering a cuda buffer as a GL buffer, and then register that GL buffer as a flex buffer?

mbechard commented 2 years ago

@warrickb did you ever find a solution to this? In my case I'd like to use buffers that are interoperable with Vulkan.

warrickb commented 2 years ago

@mbechard I think you can only create a buffer in OpenGL to use in Flex and share that across to Vulkan using extensions.