Open v-ein opened 1 year ago
Got a fix for this, going to open a PR later.
@v-ein did you ever get a fix for this uploaded? I'm running into the same issue which makes it difficult to use modern versions of DPG for async video decoding.
This seems like a pretty critical issue for any applications requiring threading to offload CPU-heavy tasks. Are there any suggested workarounds? All guidance I can find suggests using a render callback, which is no longer supported. I also found that swapping the textures in the render loop seems to trigger the deadlock more frequently in some cases.
E.g. I run into this issue when I update a texture in a background thread while also dragging a slider quickly. The application suddenly deadlocks indefinitely.
Edit: Thank you for your work on the project as well!
No, not yet. I'll see if I get some spare time to do that.
I've got more fixes related to synchronization, and hoped to get a chance some day to combine them into one and push as a single PR. Most of them are massive edits and thus are difficult to review, so my idea was to reduce the grind for reviewers. I'll see what I can do.
Offloading CPU-heavy tasks is probably best done using a background thread, like you suggested. However, DPG is not entirely thread safe, not even if we fix all the sync issues. It maintains a state (dpg.last_item()
and container stack, at the very least). Since the state is global, i.e. shared between threads, certain DPG functions are unsafe and may cause race conditions.
Each alternative has its own drawback:
start_dearpygui
, create your own rendering loop, doing heavy work between the frames. This may reduce FPS and make UI unresponsive.I hope one day I can make DPG fully thread-safe by making its state thread-local. This is a far goal, and might not be 100% doable, so don't count on that.
The first alternative you describe is the easiest to put in place, but indeed if events pile up we can skip important callbacks.
One improvement to the behaviour of this alternative could be to be able to set a flag telling to 'skip' events if a new one is available.
For example a mouse handler might prefer to skip old mouse positions and go directly to the most recent one.
Imagine a plot with heavy rendering in callback (drawing lines, circles, etc). By heavy, I mean in terms on CPU, not GPU. Doing the rendering in a callback is a good solution, as the mouse can still interact smoothly with the plot. The rendering can be performed on a hidden layer, and then when it is done, the layer is shown, and the previous one hidden (double buffering).
DearPyGui is ideal for the above scenario. The only issue is if mouse/keyboard events pile up. What do you think of introducing handler configuration options to skip redundant events in the callback queue ?
What do you think of introducing handler configuration options to skip redundant events in the callback queue ?
It's an interesting idea, though it doesn't fit well the current DPG API. I'll need to think about it. It can probably be done in Python as well, but DPG itself surely can skip events better (because it can forward-check the queue before running the handler, whereas on Python side we'd have to run all handlers first, and only then actually perform the task associated with this kind of event).
The handler could probably have kind of a "skip repetitive events" flag, but we need to clearly define what a repetitive event is. Also, I wonder if the mouse hadnler is the only handler where we can implement such filtering.
BTW it can also be done in Python with manual callback management, but #2208 will need to get fixed first.
By releasing the gil in dpg.mutex(), and encapsulating all dpg calls with dpg.mutex(), I was able to avoid any deadlock despite having several threads submitting dpg commands. This can be a simple workaround for anyone affected.
Exactly, that was my solution too. Most of the time when locking the mutex, DPG also needs to release GIL before owning the mutex.
Even this solution I have ended up with hangs. It looks like the more I add keyboard and mouse handlers, the more likely I get these... In the end I had to resort to not using the frame callback and only doing the default rendering loop, and do all drawings in the callback in a single thread.
I hope to eventually (in a couple of months) push some fixes for synchronization issues - who knows, it might help. Not going to be soon though (the fixes are completed in my local repo, but I'll need to revisit the code and also take #2275 into account).
Version of Dear PyGui
Version: 1.8.0 Operating System: Windows 10
My Issue/Question
If there are several threads calling DPG functions, there's a chance they get into a deadlock where one thread is holding GIL and waiting for the mutex, while the other does the opposite. Here's an example of how this may happen:
dpg.mutex()
and starts manipulating the widget tree (e.g. building a large table).The issue never occurs if DPG is only run on the main thread and the callbacks thread - the renderer always releases GIL before attempting to lock the mutex. The rest of the code, however, holds GIL when locking the mutex.
As soon as there's a DPG call in a thread other than those two "standard" threads, there's a chance for a deadlock. A call to
dpg.mutex()
is not necessary - the same deadlock may happen with any DPG call;dpg.mutex()
just increases chances to bump into it.To Reproduce
Note: to reproduce this, one needs PR #2004 to be merged first. Without that, the mutex is not held by DPG callers.
Steps to reproduce the behavior in general:
With the example below:
Expected behavior
No deadlocks.
Screenshots/Video
Standalone, minimal, complete and verifiable example