[macOS Ventura] new Application/Display capture drops frames, particularly in fullscreen, particularly for low FPS sources

[jcm93]

Operating System Info

Other

Other OS

macOS 13

OBS Studio Version

28.0.0

OBS Studio Version (Other)

28.0.0

OBS Studio Log URL

https://obsproject.com/logs/sdjIGgIfvysdkC6U

OBS Studio Crash Log URL

No response

Expected Behavior

macOS Screen Capture Application/Display capture not to drop frames.

Current Behavior

macOS Screen Capture Application/Display captures drop frames. The problem seems to be exacerbated greatly in fullscreen, and the problem also seems to be exacerbated when the source is rendering at a low frame rate.

Steps to Reproduce

I am currently reliably reproducing this on the current Ventura developer beta (22A5331f) on an M1 Max MacBook Pro, with no external monitors plugged in.

1. Under macOS Screen Capture, add a Display or Application capture source (see below for list of apps for which it is occurring/not occurring). Issue will be more noticeable if the source is rendering at a low frame rate.
2. Set that source application or window to fullscreen.
3. Observe many dropped frames in preview/output.

Anything else we should know?

The issue does NOT seem to be exclusive to fullscreen, but it is much more noticeable when the application is in fullscreen.

Dropped frames seem to decrease when UI and cursor movement is being rendered, and increase when it's only the source being rendered.

I have reproduced this capturing the following applications in either the Display or Application capture modes:

VLC (with low frame rate video) Parallels Dolphin

It seems to NOT be reproducible, or at least easily noticeable, in the following:

QuickTime Safari Other system applications

Attached are several videos showing the issue, with either Application or Display capture on the left, and simultaneous Window Capture (macOS Screen Capture version) on the right.

The final video shows a 60fps source, where the problem is lessened but still seems to be present.

https://user-images.githubusercontent.com/6864788/188241363-d73532c7-0e6a-492d-aadc-242a6cff5b81.mov

https://user-images.githubusercontent.com/6864788/188241389-43e08ec6-96df-4079-ab43-4c6664ab0d36.mp4

https://user-images.githubusercontent.com/6864788/188241413-cf6ef7e4-3ee0-4b8a-9dee-b7812bc283fd.mp4

https://user-images.githubusercontent.com/6864788/188241426-75785e8c-86d4-47b0-a813-5e3630eb4321.mp4

[PatTheMav]

@gxalpha can you cross-check this on your macOS 13 machine?

When I put a window in full screen on macOS 12, the preview is frozen (so 0 fps) and I have to switch to the corresponding window to get frame input, but then it doesn't drop frames for me (tested with OpenEMU and PSX emulation).

[gxalpha]

I cannot reproduce this by on macOS 13 Beta 7 by playing a 5 FPS video file in VLC fullscreen using SCK Display Capture.

[jcm93]

Hm, odd. Here's a video from my machine of fullscreen VLC with a 24fps video. I am still on beta 6 though, could this be somehow fixed in beta 7? I couldn't find a 5fps video to easily test. Would you mind maybe testing with the second attachment here?

https://user-images.githubusercontent.com/6864788/190484630-05cd81bf-6fed-4912-bfad-3d1b2681181a.mp4

https://user-images.githubusercontent.com/6864788/190484681-0c97498c-c8ec-4e39-aba0-7ed9566486e6.mp4

[PatTheMav]

Just did a test run with that file on macOS 12.6 and macOS 13.0 Beta 7:

https://user-images.githubusercontent.com/9436503/190641886-5a1e3bdc-0965-4a4d-bbb7-5bc846e7543e.mp4

Apart from some minor frame pacing issues I don't see noticeable frame drops.

[jcm93]

This is quite odd. The issue is still occurring for me on macOS 13 beta 8:

https://user-images.githubusercontent.com/6864788/192110510-099a96f0-d536-43f6-bd33-0b1032faea23.mp4

Just confirmed that it occurs with IINA as well, so it must be something unique to my setup? Unless it's an issue that disappeared with beta 7 and reappeared in beta 8.

[jcm93]

Figured something out. The issue disappears if I set my screen's refresh rate to 60 Hz instead of 'ProMotion.' Is your display 60 Hz perhaps @PatTheMav ?

edit: Checked to see if it occurs booting in safe mode; it does...

[PatTheMav]

Figured something out. The issue disappears if I set my screen's refresh rate to 60 Hz instead of 'ProMotion.' Is your display 60 Hz perhaps @PatTheMav ?

edit: Checked to see if it occurs booting in safe mode; it does...

As a matter of fact I only have a Studio Display available which is locked at 60fps, I have no ProMotion display available to me.

@RytoEX, @derrod - which one of you got the 14-inch or 16-inch MacBook Pro?

[RytoEX]

Figured something out. The issue disappears if I set my screen's refresh rate to 60 Hz instead of 'ProMotion.' Is your display 60 Hz perhaps @PatTheMav ? edit: Checked to see if it occurs booting in safe mode; it does...

As a matter of fact I only have a Studio Display available which is locked at 60fps, I have no ProMotion display available to me.

@RytoEX, @derrod - which one of you got the 14-inch or 16-inch MacBook Pro?

Not I. I have a 13-inch MBP.

[derrod]

@RytoEX, @derrod - which one of you got the 14-inch or 16-inch MacBook Pro?

I have a 14 inch one, but so does @gxalpha. However I'm not on macOS 13 yet.

[gxalpha]

Mine is 16.

[PatTheMav]

So at this point the issue cannot be reproduced by maintainers - I do think that ProMotion might have a role in this, but without reproduction we're flying blind here.

I do wish Apple had a reference implementation of an SCK receiver so OBS and its implementation could be ruled out and such issues reported directly to Apple.

[Developer-Ecosystem-Engineering]

We were able to reproduce it, believe its a platform issue. As a short term workaround, using just "Display" capture instead of "Display, Including Windows" should provide higher FPS in this scenario.

@PatTheMav for a reference implementation, we use the sample app , if the problem occurs there and in OBS, its a good indicator it maybe on the platforms end. Not 100% but it's a good test.

[jcm93]

I'm reluctant to say this issue is resolved in Sonoma, since I'm still seeing SCK drop frames, but behavior does seem to have changed in macOS 14. An app in fullscreen captured with display or application capture no longer sees drastic frame drops (compared to the non-fullscreen behavior).

[MisutaaAsriel]

I actually started experiencing this with Sonoma. — Desktop or Application capture are fine, but Window capture causes severe frame drops, to a point macOS itself begins to lag out.

Disabling the preview helps to stabilize things, but it never hits the max framerate as set in settings and the OS animations are noticeably choppy still until SCK is disengaged or switched to Desktop/Application view.

Note that it doesn't always occur. Primarily when content is in motion or otherwise animated does the issue occur. When the window is static, performance is at expected levels. But… this is practically pointless unless you are streaming a spreadsheet.

It's practically unusable currently.

(Device BTW is a MacBook Air M2 with 24GB RAM)

[gxalpha]

@MisutaaAsriel thats a known regression with Sonoma that we reported to Apple a while back (FB12277154(?) for reference). However I'd imagine it can’t hurt to file an additional FB with your specific steps to reproduce (and if you do so, also consider posting the number here for better visibility).

[jcm93]

I have a feedback in for the Sonoma Window Capture issue as well, FB12326079, submitted June 12. My 2c is that it's highly dependent on captured window size as well. Mine has not been triaged or updated at all so I don't know what the status is. Probably worth spinning into a separate issue? Though it's not an OBS issue, it's nice to have tracking given that Apple feedbacks are a black box.

[PatTheMav]

I have a feedback in for the Sonoma Window Capture issue as well, FB12326079, submitted June 12. My 2c is that it's highly dependent on captured window size as well. Mine has not been triaged or updated at all so I don't know what the status is. Probably worth spinning into a separate issue? Though it's not an OBS issue, it's nice to have tracking given that Apple feedbacks are a black box.

It's usually fine to just post Feedbacks on any issue you encounter - as Apple does not publicly list known or already-reported issues it's probably expected that duplicates might be reported.

[MisutaaAsriel]

However I'd imagine it can’t hurt to file an additional FB with your specific steps to reproduce (and if you do so, also consider posting the number here for better visibility).

FB12770653 is my feedback. I specifically put it under 3rd Party Apps and noted it was an issue with Screen Capture Kit.

I submitted the text as follows:

Window Capture with Screen Capture Kit is severely laggy

When recording a window from a Screen Capture Kit enabled application, system performance is severely degraded, dropping to sub-30FPS framerates. Likewise, capture quality is equally reduced to sub-20FPS. This only effects window capture, not application nor desktop capture, and is increased in effect based on the amount of "activity" (different frames) in the target window (e.g. if the application is static and unchanging, performance returns. If anything changes, such as scrolling, changing content, animations, etc., performance is degraded by the amount of movement.)

[gxalpha]

@jcm93 @MisutaaAsriel are either of you still experiencing any of the problems described here (both in the original issue as well as the ones brought up in the comments) in the latest Beta of macOS 14 (23A5337a at the time of writing)? Afaik the current theory is that things should be fixed there.

[jcm93]

For my part, in the latest macOS, I do not notice the original issue in this thread, or a substantial amount of dropped frames in any capture mode.

It seems safe to me to at least close this issue, since the specific behavior from the OP here is definitely no longer occurring. If I find evidence of SCK dropping a substantial amount of frames in other situations, seems to me like it'd be better in a new thread.