search

niessner / Opt

Opt DSL
Other
256 stars 68 forks source link

Support running Opt on non default cudaStream #145

Open gerwang opened 5 years ago

gerwang commented 5 years ago

I want to launch another CPU thread (call it CPU thread B) and use Opt on that thread, and I don't want functions like Opt_ProblemSolve to block GPU execution invoked on CPU thread A.

Does Opt support running Opt_ProblemSolve on different cudaStream? If Opt can only run on default stream, then if I schedule all GPU tasks invoked by thread A on a non-default cudaStream, can two CPU threads run without blocking each other?

By the way, does Opt call cudaDeviceSynchronize internally? If so, I'm afraid those two threads will block each other.

Mx7f commented 5 years ago

Opt does not currently support non-default cudaStreams. There are three levels of support we could offer:

  1. Internally create and use a non-default stream internally for all operations.
  2. Expose the stream parameter for reading via the C API,
  3. Expose the stream parameter for writing via the C API (probably in Opt_InitParams).

2 and 3 worry me because we'd need a portable way to expose the stream in the C API, which currently includes no headers, especially not cuda headers. We could probably be fine passing streams as (void*) though.

The first option is easier, and likely sufficient for your purposes. This will require non-trivial, but straightforward changes to API/src/solverGPUGaussNewton.t and API/src/util.t to use a stream when possible. This includes making many cudaMemcpy's into async to avoid unneccessary synchronization.