OBS Initializes OpenGL on wrong Graphics card at startup if external monitor is connected

[cdsimpkins]

Operating System Info

macOS 13

Other OS

No response

OBS Studio Version

30.0.0-rc2

OBS Studio Version (Other)

No response

OBS Studio Log URL

Multiple Logs Uploaded. See repro Steps

OBS Studio Crash Log URL

Multiple Logs Uploaded. See repro Steps

Expected Behavior

• Preview Should continue to render properly when I move the OBS window to a secondary screen.
• Application Should not crash when I move it back to built in screen.

Current Behavior

• Preview Freezes with a sort of picture-in-picture view of the stream content
• Moving the application window back to the built-in display will crash the application

Steps to Reproduce

Scenario 1 (Crash)

1. Plug in external display via dock (Mine is a USB-C Dock that requires DisplayLink Manager software to use)
2. Launch OBS
   • OBS Selects Integrated GPU to render OpenGL
3. Move OBS window to external display
   • Preview will Freeze
4. Move OBS Window back to built in display
   • OBS Will Crash

OBS Log Crash Log

Scenario 2 (No Crash)

1. Ensure no external monitors are plugged in
2. Launch OBS
   • OBS Selects Dedicated GPU to render OpenGL
3. Plug in external display
4. Move OBS Window to External Display
   • I see a brief flash where the preview redraws and resizes, but it's only a Frame, and it does not freeze.
5. Move OBS Window back to builtin Display
   • I see a brief flash where the preview redraws and resizes, but it's only a Frame. OBS Does not crash.

OBS Log

Anything else we should know?

2019 Macbook Pro 16" MacBookPro16,1 2.6GHz 6-core intel Core i7 AMD Radeon Pro 5300M Graphics Processor w/ 4gb VRAM Intel UHD Graphics 630 w/ 1500MB VRAM 32GB DDR4 Ram

I was originally writing this up as a different issue but I found Scenario 2 while going back to grab more information after having unplugged my machine.

If you look at the log from Scenario 1, you'll see where it initializes OpenGL On the integrated graphics card (but in that case the external displays are connected and they require use of the dedicated GPU).

09:33:37.636: Initializing OpenGL...
09:33:37.702: Loading up OpenGL on adapter Intel Inc. Intel(R) UHD Graphics 630
09:33:37.702: OpenGL loaded successfully, version 4.1 INTEL-20.6.4, shading language 4.10

The logs for Scenario 2 show that with no external displays connected, OBS initializes OpenGL using the Dedicated GPU:

09:43:56.106: Initializing OpenGL...
09:43:56.158: Loading up OpenGL on adapter ATI Technologies Inc. AMD Radeon Pro 5300M OpenGL Engine
09:43:56.158: OpenGL loaded successfully, version 4.1 ATI-4.14.1, shading language 4.10

and then the application can freely move between monitors if they have been plugged in after OBS starts up.

I have found these repro steps work on every latest major release of OBS below:

• 25.0.8
• 26.1.2
• 27.2.4
• 28.1.2
• 29.1.3
• 30.0.0-rc2

I have only pulled logs for 30.0.0-rc2, so I do not know if it's the exact same issue, but the repro steps produce exactly the same behavior.

Now that I have discovered a workaround (unplug displays, then start OBS), this seems more like a minor annoyance, but I would expect this issue is pretty common. Let me know if you need more information or how I can help. I would be glad to see this issue fixed!

[PatTheMav]

The observation is correct - OBS does not implement functions to detect a display change and identifying the associated GPU with that display (see https://developer.apple.com/documentation/metal/gpu_devices_and_work_submission/multi-gpu_systems/getting_the_gpu_that_drives_a_view_s_display?language=objc).

We would need to react to the NSNotification that tells us about that change and then re-initialise the renderer (usually macOS taps into the discrete GPU once an internal display is connected for the entire system and switches back to the internal GPU when all external displays are disconnected, but that's not a hard rule).

I don't use Intel-based Macs anymore, so I'd invite anyone who still does and has a integrated/discrete GPU situation to implement and test handling this notification.

[cdsimpkins]

I've been doing more research and testing on this issue. I've also found a second workaround, an additional (potentially related?) problem, and some more information regarding the interaction between DisplayLink and OpenGL

TL;DR: While I think the crashes could be fixed, I still couldn't fully utilize my setup as a streaming host due to the combination of OBS and Apple/Mac Limitations.

2nd workaround

If I launch OBS with my laptop lid closed, OBS still says it initializes on the integrated GPU, but the app can still freely move around between the two external monitors. I have to keep the lid closed, though, or the crash occurs again. The Additional Problem described below does not occur when I have the lid closed. Everything seems to be operating normally, I'm just down one monitor and mildly afraid that I'm choking my laptop with airflow and heat retention.

Additional problem

If I use the first workaround (Launch OBS with the dock unplugged, then plug them in), when I try to use the Fullscreen Projector (Preview) option on one of the external monitors, the preview window will show a blank screen. When I start the preview, the log begins to fill with the same message as before, but there is no crash:

...
13:20:38.395: glBindFramebuffer failed, glGetError returned GL_INVALID_FRAMEBUFFER_OPERATION(0x506)
13:20:38.412: glBindFramebuffer failed, glGetError returned GL_INVALID_FRAMEBUFFER_OPERATION(0x506)
13:20:38.429: glBindFramebuffer failed, glGetError returned GL_INVALID_FRAMEBUFFER_OPERATION(0x506)
...

Additional Information

• According to OBS Documentation, Hardware Encoding on an intel-based Mac is explicitly not supported for Streaming. So It seems like even if the visual display bug is fixed, I still cannot actually use the Discrete GPU to render frames for streaming.
• PrusaSlicer, which uses OpenGL to render 3d views, also has some issues when trying to display the 3d view on a monitor connected via DisplayLink. It doesn't crash, but the 3d view stops rendering and cannot be interacted with.
• Even with someone developing OpenGL Applications for fun has had issues with DisplayLink-connected monitors and OpenGL.

It seems I just have an incompatible setup if I want to make full use of this laptop as a streaming host. I could try getting a Thunderbolt3 cable and changing my hardware around a bit to avoid the dock, but My two monitors are an old dumb acer monitor and a Dell monitor that supports DisplayPort Alt-mode. Apple does not support Multi-Stream Transfer with Displayport Alt-Mode so I would still be stuck with only two monitors in that case. And even if I could solve the display limit problem, I still cannot make OBS use my discrete GPU for rendering.

Sorry for the book, I am mostly just writing down all of this in case anyone else finds this issue while looking for answers to the same problems I have had.

[PatTheMav]

Some clarifications:

• Encoding is handled separately from compositing - the main reason you cannot use VideoToolbox for streaming is that Apple added constant bitrate (CBR) support to Apple Silicon only, but it's what the majority of streaming services requires as their ingest format.
• macOS abstracts the actual devices uses behind NSOpenGL and VideoToolbox, so it can be that the same device is used for each (it's the most probable scenario) but it doesn't have to be and we have only limited control over that (e.g. there are VideoToolbox encoder IDs specific to the T2 chip found in Intel-based Macs), but we rely on macOS giving us the correct device when we ask for one
• The choice of adapter is usually important on Windows to ensure that the captured frames of a game don't need to be shuffled from the originating GPU to another GPU doing the compositing for OBS - having both run on the same GPU is usually more efficient and less taxing on the system

It could very well be that we need to get a new NSOpenGLContext when the device driving the screen has changed, but the DisplayLink-specific error reminds me of a similar error we've seen with virtual displays generated by BetterDisplay, but that is internal to how these tools implement Displays and macOS handles them.