search

NVlabs / FoundationPose

[CVPR 2024 Highlight] FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects
https://nvlabs.github.io/FoundationPose/
Other
1.36k stars 176 forks source link

Multithreading and multiprocessing issues #187

Closed GitMichaelR closed 1 month ago

GitMichaelR commented 2 months ago

Hello,

I run two threads with one foundationpose model each. However when they are both executing the register function and within it the erode depth or bilateral_filter_depth function, an error occurs because of warp. So currently, as a "fix", I am running the register function wit a gpu_lock which is basically as fast as not using multithreading. Is there a workaround?

Thank you!

GitMichaelR commented 2 months ago

And with multiprocessing and the start method "spawn" I get an error like: ".../multiprocessing/reduction.py", line 60, in dump ForkingPickler(file, protocol).dump(obj) File "stringsource", line 2, in zmq.backend.cython.context.Context.__reduce_cython__

Without explicitly defining a spwan method I get: glctx = dr.RasterizeCudaContext() ......... RuntimeError: Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method.

wenbowen123 commented 2 months ago

what if you set the start method to "spawn"?

GitMichaelR commented 1 month ago

Like I said in the comment before, I get the error:

".../multiprocessing/reduction.py", line 60, in dump ForkingPickler(file, protocol).dump(obj) File "stringsource", line 2, in zmq.backend.cython.context.Context.reduce_cython

Can you by any chance provide a working minimal multithreaded example?