RetroAchievements / RAIntegration

The DLL responsible to integrate emulators with RetroAchievements.org
https://retroachievements.org
MIT License
95 stars 22 forks source link

prevent infinite recursion trying to invoke to inaccessible UI thread #1081

Closed Jamiras closed 4 months ago

Jamiras commented 5 months ago

https://discord.com/channels/310192285306454017/1231413250915307580/1231413250915307580

RAMeka has two message pump threads - one for the Allegro window (main window), and one for the console window. The DLL has to be attached to the Allegro window for the overlay functionality, but the editor windows have to be parented to the console window or shortcuts don't work because the Allegro message pump doesn't call TranslateMessage.

Because the DLL is only told about the Allegro window, it assumes the message pump for the Allegro window is the UI thread. When it tries to invoke to the UI thread, the messages are sent to the console window because that's where the child windows are being managed. Some of the methods that try to redirect to the UI thread say "if I'm not on the UI thread, call me from the UI thread". However, because the DLL is looking for the UI thread belonging to the Allegro window, and the method keeps getting called from the console window thread, it just keeps trying to switch threads, and eventually the stack overflows and the application crashes.

Solution: If the thread identified as the UI thread is not the thread that a child window is being created from, disable the ability to invoke to the identified thread. This should only affect RAMeka.