Screen Savers broken on 2+ monitor system in macOS Sonoma 14.0

[xmddmx]

This is a follow-up thread to https://github.com/JohnCoates/Aerial/issues/1286

Initial testing with macOS Sonoma (first beta, 23a5257q) suggests the problem has worsened again:

• with 2 screens, I am seeing the same problem we saw back in early 13.3 betas: screen 1 is blank, screen 1's content is showing up mis-sized on screen 2. And, it is now back to happening 100% of the time (20 out of 20 failures, using Aerial, using HotCorner)
• similar problem with other third-party screensavers (such as https://iscreensaver.com ) so this is again not a bug in Aerial.
• I'm also seeing the same problem with first-party Apple screensavers (such as Message and Shifting Tiles)!

Conclusion: the first beta of Sonoma 14.0 is broken, and seems to be as broken as early betas of Ventura 13.3.

[glouel]

Can confirm all of the same FYI.

[glouel]

Also, side note, while they added some "video wallpapers" based on Aerial, there's no screensaver. When you get out of the screensaver (whatever it is), the lock screen plays a video for 3 seconds and stays still.

That's... not what they showed with fanfare during the presentation (seemless playback between screensaver and desktop background, which is definitely an impressive syncing effort).

[xmddmx]

FYI, I've re-submitted this as FB12240576, referencing my prior feedbacks FB12023530 and FB12179420, and also fixed the TItle of this Issue (it said macOS 13.4, but should obviously be 14.0)

[DSBlackHeart]

Version 14.0 Beta (23A5257q)

Aerial still does not function.

[x86txt]

I have also reported it to Apple via the Feedback Assistant. Even the stock included Aerial Video Wallpapers don't work, they just show up as still wallpaper images. Everyone should report it, as the more reports to Apple the more attention it will get. Use the Feedback Assistant that came with the Sonoma Beta. If you don't have it installed, here are instructions for it: https://support.apple.com/guide/feedback-assistant/intro-to-feedback-assistant-fba2e39e53f5/mac

[glouel]

Even the stock included Aerial Video Wallpapers don't work, they just show up as still wallpaper images.

But to be clear, that's how Apple implemented them. They are not video wallpapers as in, they are not intended to playback all the time (like with Companion). They just play when you login basically, then slowdown to "become" a fixed wallpaper. That's how Apple implemented them. There's not (at least not yet) a screensaver with those dynamic wallpapers... it's weird right now. You can animate them for like 5 seconds by going to your Apple menu top left and pressing "lock screen" (2nd option from the bottom).

Yet they clearly talk about screensavers here : https://www.apple.com/macos/sonoma-preview/

New slow-motion screen savers of breathtaking locations from around the world look beautiful on your large Mac display. When you log in, they seamlessly become your desktop wallpaper.

The log-in thing is here, as long as it's from boot up, or when you come back from display off (basically, a blank lock screen). If you somehow run any screen saver, the (traditional) lock screen will appear on top of the screensaver.

So it's - at the moment - a "lock screen screen saver". I doubt this is their full plan. If it is, I mean... I have a hard time processing this, not gonna lie !

[glouel]

I'll point to this update here for reference : https://github.com/JohnCoates/Aerial/issues/1286#issuecomment-1583035720

[RandyMacmini]

Can confirm all of the same FYI.

I am running 3 monitors and all 3 of mine are using screen savers. I have different ones on each screen.

[glouel]

I am running 3 monitors and all 3 of mine are using screen savers. I have different ones on each screen.

Do you mean different screensavers or different Aerial videos on each screen using Aerial ? Not sure I follow what you mean here.

[xmddmx]

Sonoma beta 2 is out, and the Screen Saver System Settings has changed a lot.

1. I am now seeing Apple's "Aerial" screensavers showing up in the Screen Saver settings - I was not seeing any of these in beta 1.
2. there is a new 'Show on all displays' option (which when Off, gives you a choice of which monitor to show the screensaver on.
3. behavior is superbly buggy, both with Apple screensavers and third party. I'm seeing new bugs where the legacyScreenSaver engine seems to get stuck running in the background, screensavers are not playing on 2 monitors correctly, etc.

Here's what the System Settings pane looks like now:

[xmddmx]

If you choose one of the Apple "Aerial" screen savers, you get a new Show as wallpaper option:

[xmddmx]

A new bug I've never seen before: when using your Aerial (not Apple's version) the screensaver actually worked on both screens, but then I was unable to exit! No keypresses or mouse clicks or movement would exit.

About 30 seconds later, the screensaver finally exited (and I heard some 'beep' noises) suggesting the whole system was lagging. Now legacyScreenSaver appears to have hung:

In short - lots of new stuff, much of it broken.

[xmddmx]

And when running on a 2 monitor system, with 'Show on all displays' enabled, behavior is again erratic: sometimes it plays only on screen 1, sometimes only on screen 2 (but with screen 1's content), and on rare occasion it will play on 2 screens nor mally, but I've only seen that about 1 out of 10 times.

[glouel]

Can confirm all this, including the new weird bugginess. Also note that if you go into desktop backgrounds (is it now called wallpaper in settings ??) you have the invert switch, use as screen saver :

The important bit is, screensavers can now be run independently on each screen, and in theory you can run 2 different ones simultaneously on different screens (at least that's what they are going for). I think that the new ability to set a separate screensaver on different screen is likely the root cause of all our issues, they might have started implementing stuff around this a while back.

Right now if I set Aerial to all screens in 2 monitor configuration, I get it on either one of them, primary or secondary. The bug previously never displayed on primary. If I set "individually" on both screens, I get the same thing. It's insanely buggy in any case.

Regarding their implementation of aerial screensavers, I think they simply use the desktop window and animate/stop it. they don't create a new separate view. This is what they did previously with the lock screen thing, they just animate the desktop and stop it.

It's unlikely we'll get the ability to make "dynamic wallpaper/screen savers" soon as 3rd party, which obviously sucks.

[DSBlackHeart]

Hey is it just me or has this thing not worked even close to right for more than a year now ?

[glouel]

@DSBlackHeart I first reported the bug february 23rd, this was 13.3 beta where they broke it.

Back on topic, if I set a dynamic background (but not as screensaver), and launch screensaver (in that case aerial), when I come back to the desktop, I get the slowed down animation and...

Did they really implement this in LEGACYSCREENSAVER 😵‍💫

I can see how that would break legacyScreenSaver for "regular" uses. And that CPU usage for a still frame, seriously (and I have to force quit them if I switch back to regular wallpaper) !

[xmddmx]

Hey is it just me or has this thing not worked even close to right for more than a year now ?

More like 4 months - it's only been since 13.3 betas which started happening in March. Feels like a year though!

The important bit is, screensavers can now be run independently on each screen, and in theory you can run 2 different ones simultaneously on different screens (at least that's what they are going for).

Maybe? they don't really have a good UI for that, as it's only "Run on main screen" which if off, lets you choose 1 monitor to run on. I can't see any way to intentionally choose N arbitrary screensavers for N monitors, can you?

I think that the new ability to set a separate screensaver on different screen is likely the root cause of all our issues, they might have started implementing stuff around this a while back.

good theory

[xmddmx]

Did they really implement this in LEGACYSCREENSAVER 😵‍💫

Yeah, I'm seeing that too. That's a real WTF.

[glouel]

The important bit is, screensavers can now be run independently on each screen, and in theory you can run 2 different ones simultaneously on different screens (at least that's what they are going for).

Maybe? they don't really have a good UI for that, as it's only "Run on main screen" which if off, lets you choose 1 monitor to run on. I can't see any way to intentionally choose N arbitrary screensavers for N monitors, can you?

It's very clunky but it works just like on the desktop backgrounds side, whatever you set is persisted, if I setup Aerial on one and switch to the other screen, set monterey there, if I pick the first screen back it shows Aerial.

And in that case I can confirm it runs both (and in that case, weirdly enough, fine).

[xmddmx]

I feel like with Sonoma beta 1, some people were talking about the new screensavers. But most of us were not seeing any new screensavers, only wallpaper. This was very confusing to me, and I was wondering if they were confused.

Now in beta 2 I'm seeing actual screensavers.

Could it be possible that poeple were seeing the screensavers in beta 1, but it was only showing up for some people? (a certain region, model of mac, GPU, CPU or something else?)

[glouel]

I'm 100% sure people just got confused with the lock screen and Craig saying screensavers ;)

In beta 1, if you set the desktop background, you got the animation at login, back from sleep, or pressing lock screen in your Mac menu bar top left. I doubt they A/B tested anything, just people not understanding the difference or reusing Apple's marketing.

9to5mac was infuriating about this for example.

[xmddmx]

It's very clunky but it works just like on the desktop backgrounds side, whatever you set is persisted, if I setup Aerial on one and switch to the other screen, set monterey there, if I pick the first screen back it shows Aerial.

OK, I see what you are saying, and it kinda works.

Unfortunately, this means third-party screensaver authors now have to test for the difference between

• one screensaver, set to 'Show on all displays'
• the same screensaver, set to 'Show on one display', but set to show on all displays using this confusing UI.

as well as other possible issues such as

• two different versions of the same screensaver, both playing on different displays.
• etc.

Yikes!

[glouel]

It's very clunky but it works just like on the desktop backgrounds side, whatever you set is persisted, if I setup Aerial on one and switch to the other screen, set monterey there, if I pick the first screen back it shows Aerial.

OK, I see what you are saying, and it kinda works.

Unfortunately, this means third-party screensaver authors now have to test for the difference between

• one screensaver, set to 'Show on all displays'
• the same screensaver, set to 'Show on one display', but set to show on all displays using this confusing UI.

Yeah, I think it may not change much, we already had different situations in macOS where your plugin was launched by one process and ran on 2 windows, or launched by 2 separate processes. I know I have code about that laying around in Aerial, I can never remember what the current situation is. The tiny preview in System Preferences was also sometimes sharing the same process (I think they no longer do that, if I had to guess, but not sure).

as well as other possible issues such as

• two different versions of the same screensaver, both playing on different displays.

Usually this goes very wrong, it's worth trying putting one version in /Library and one in ~/ and set them both, see what happens. System Preferences for example couldn't handle this if you put 2 versions, itonly loaded the UI bundle from one, which lead to hilariously hard crashes to debug if you opened the second. Now I just don't let people put Aerial in /Library for that reason ;)

Adding a bit on the legacy thing, it's late and I'm a bit too tired to investigate but I'm starting to wonder if what they do is, run the unlock screen animation through legacyScreenSaver, and at the same time set the last frame of the animation as a "picture" desktop background proper, and hide (but not kill) the legacyScreenSaver process called Wallpaper. I'm saying this because force quitting it just results in nothing changing visually.

Edit : Thankfully all those changes are brilliantly documented... https://developer.apple.com/documentation/screensaver 😩

[xmddmx]

Yeah, I think it may not change much, we already had different situations in macOS where your plugin was launched by one process and ran on 2 windows, or launched by 2 separate processes. I know I have code about that laying around in Aerial, I can never remember what the current situation is. The tiny preview in System Preferences was also sometimes sharing the same process (I think they no longer do that, if I had to guess, but not sure).

So, with recent versions of iScreensaver, which are written in Swift, there's a nice multi-screen syncrhonization feature, so when transitioning to another asset (image, video, 3D GLTF file...) each screen would talk to the other one and try to wait until all were ready, so they can all run the next effect simultaneously. When it works it's pretty nice.

Because legacyScreensaver did run all instances in the same process, this was fairly easy to do, by just using Swift globals. Or at least it seemed easy and worked on Intel, but when Apple Silicon came around, the memory model changed and having mutliple threads accessing globals no longer worked, as described here: https://developer.apple.com/documentation/apple-silicon/addressing-architectural-differences-in-your-macos-code

In any case, it was solved by adding some protection around the memory access using the synchronized directive or similar, see https://stackoverflow.com/questions/24045895/what-is-the-swift-equivalent-to-objective-cs-synchronized

With Sonoma, this doesn't seem to be working, so I need to go back and check this assumption that all screens are running in different threads but the same process. Maybe it's not true any more?

[glouel]

Super interesting, I think it's quite possible they now run multiple instances of legacyScreenSaver, would be interesting to look at.

Thinking about it, I'm pretty sure I'm relying on sharing a playlist object on Aerial too, for the spanned effect where I play a single video across multiple screen (I don't use mutexes though). It's late so I'll try to check more tomorrow. Could be a huge mess indeed 😩

[xmddmx]

OK, I can clearly see one bug - here's a sample log from iScreensaver being launched on Sonoma beta 2 when set to Show on all displays

Process:Thread  Library         Message
1191: 0xaaf4    iScreensaver    ########## iScreensaver.init frame=(0.0, 0.0, 1680.0, 1050.0) isPreview=false
1191: 0xaaf4    iScreensaver    ########## iScreensaver.initalize 0x00007fca6c762830
1191: 0xaaf4    iScreensaver    ########## iScreensaver.initalize isPreview=false previewMode=false debugFlag=false hudTestFlag=false
1191: 0xaaf4    iScreensaver    ### getScreenNumber defaulting to 0
1191: 0xaaf4    iScreensaver    ###  screen=[0] globalTestVar=1 myAddress=0x00007fca6c762830
1191: 0xaaf4    iScreensaver    #   [0] screenNumber = 0
1191: 0xaaf4    iScreensaver    #   [0] mainScreen = false
1191: 0xaaf4    iScreensaver    ###  [0] hasCongfigureSheet 0x00007fca6c762830
1191: 0xaaf4    iScreensaver    ###  [0] hasCongfigureSheet 0x00007fca6c762830
1191: 0xaaf4    iScreensaver    [0] # draw 0x00007fca6c762830
1191: 0xaaf4    iScreensaver    ########## iScreensaver.init frame=(0.0, 0.0, 1440.0, 900.0) isPreview=false
1191: 0xaaf4    iScreensaver    ########## iScreensaver.initalize 0x00007fca6c77d820
1191: 0xaaf4    iScreensaver    ########## iScreensaver.initalize isPreview=false previewMode=false debugFlag=false hudTestFlag=false

Translating this gibberish:

• the ScreenSaverView.Initialize call is being called once for each screen
• screen[1] is being initialized first, which is unusual, but isn't a big problem, however...
• the frame is wrong - 1680x1050 is my second monitor, and it's definitely not at coordinates 0,0
• this incorrect X,Y coordinate is confusing our code (which figures out which monitor it's on based on the X,Y,W,H of the frame) so it's mis-detecting and defaults to Screen[0].
• Next, the real Screen[0] is initialized, which has the correct coordinates (0.0, 0.0, 1440.0, 900.0) so it detects it's on screen zero
• at that point, everythign is Fubar'd because there are two Screen[0]s running, and they step on each other.

Other thoughts:

• I may be misreading this, but the debug logging shows that not only are the two screens being run from the same process (1191) they are also being run from the same thread (0xaaf4). This doesn't seem right to me, and perhaps could be part of the OS bug we are seeing? I would assume that each screen should be running in it's own thread, right?
• is there another way to figure out which actual screen we are on? our code uses the x,y,w,h of the frame, but since x,y are wrong, this isn't working...

[xmddmx]

Another test, this time with the System Settings / Screen Saver pane open shows that the small screen saver Preview window is in fact run in a different process:

Process:Thread  Library         Message
1251: 0xc6cb    iScreensaver    ########## iScreensaver.init frame=(0.0, 0.0, 143.0, 80.0) isPreview=true
1268: 0xc7c3    iScreensaver    ########## iScreensaver.init frame=(0.0, 0.0, 1440.0, 900.0) isPreview=false

Note: when isPreview=true, the PID and ThreadID are different.

So, in Sonoma, screensavers on multiple monitors are run in the same process and thread (!?) during normal operation, but the screen saver settings Preview mode runs in its own process.

I've done some more testing and I think this behavior is similar in Ventura.

Is it possible that the main bug in Sonoma beta 2 is that it's giving us the wrong X,Y coordinates for Screen[1], Screen[2] etc.? Perhaps a workaround is possible.

[xmddmx]

Also, on Ventura on a 2 monitor system, screen[0] is initialized first, followed by screen[1] which seems more logical.

Also, both instances get the correct coordinates (the .frame.x and .frame.y are not zeros for screen[1] and higher...)

Ventura:

0x21642 6202    iScreensaver    ########## iScreensaver.init frame=(0.0, 0.0, 1440.0, 900.0) isPreview=false    default legacyScreenSaver
0x21642 6202    iScreensaver    ###  screen=[0] globalTestVar=1 myAddress=0x00007fcaca005940    default legacyScreenSaver
0x21642 6202    iScreensaver    ########## iScreensaver.init frame=(1440.0, 0.0, 2560.0, 1440.0) isPreview=false    default legacyScreenSaver
0x21642 6202    iScreensaver    ###  screen=[1] globalTestVar=2 myAddress=0x00007fcaca008440    default legacyScreenSaver

[glouel]

My understanding of the original bug was that as part of it, primary screen was misplaced on last screen so the messed up coordinates are part of that, but as far as I know we can’t change them. This is probably more complicated now I’ll try to dig a bit but I think it’s going to be a long wait for beta 3 🫣

[glouel]

Random observations :

• I found, while not using the new desktop stuff, and only using Aerial, that legacyScreenSaver (Wallpaper) was actually Aerial that kept running in the background. My guess, 3rd party screen saver end up on the same path and they get "thrown" as wallpapers and kept running (which would be great) but that just doesn't work. I don't expect this to actually happen to be clear, it's just IMO their own implementation of mixed desktop background/screen saver, and we get caught in the middle. Reported as FB12428045

• As you pointed out, we are initialised incorrectly with coordinates that are always zeroed in for origin. This can happen even when we end up running on a screen not zeroed in (I have 2 screens side to side, main is left, if we run on right, we still get it). As far as I can see, in my testing, most of the time we run on one screen or the other, if I set different sizes for my screens, we end up with various configurations, but mostly never the correct view on the correct screen. This is a variant of the original multimonitor bug, except we don't always end up on the last screen as before, now it's random. I'm still working on how to report those changes, but it's terrible.

• Most of the time it shows the wrong preview in settings, I get the Ventura one running :

Funnily enough, when you press the preview button, things work correctly (on multiple screens even) ! A refreshing bug. Options button is not there though. Selecting another screen saver and selecting Aerial back fixes it. Reported as FB12428208

[glouel]

Adding this one which is probably what happens for bug 1 above :

I was pretty sure I was detecting saverwillstop events to pause the screensaver, and I did, yet we are playing. I have a hard time processing this but I believe we are somehow restarted as (unshown) wallpapers as Aerial keeps playing in the background, somehow, despite me pausing it when receiving the event.

[xmddmx]

More Sonoma Screensaver Bugs:

• in Preview mode, mouse events are being received, which is not true in older OS's. Not necessarly a bug, but if your code assumes mouse events only happen while running fullscreen, this could lead to issues.
• I confirm what you are seeing: the 'Preview' button to start the screensaver sometimes works differently than HotCorner or timeout.
• the sequence of calls to startAnimation/stopAnimation (etc.) seems to have changed when using the Options panel, but I haven't diagnosed just what the changes are.
• iScreensaver handles screensaver Options using a helper app. This launches OK, but now in Sonoma, if you hide the Helper app window, and then click the screensaver 'Options' button a second time, the .saver bundle tries to push the helper app frontmost using this code:

           let apps = NSRunningApplication.runningApplications(withBundleIdentifier: childBundleID)
            if let helperApp = apps.first {
                helperApp.activate(options: [ .activateIgnoringOtherApps ])
           }

This is no longer working, but I'm not sure why.

[xmddmx]

Most of the time it shows the wrong preview in settings, I get the Ventura one running :

I'm seeing something similar, or perhaps a variation?

• if you have a screensaver selected, the correct screensaver shows up in the small Preview mode. If you click the 'Preview' button, that same screensaver runs fullscreen.
• however, you can close system preferences, and activate the screensaver (using HotCorner, or Timeout...) and the OS will launch the wrong screensaver. In my case, looks like I have two screensavers with different filenames, but the same internal bundleID. I wonder if the bundleID is confusing the OS?

[shivajreddy]

Version 14.0 Beta (23A5257q)

Aerial still does not function.

Dinesh that you with all the apple monitors?

[DSBlackHeart]

Version 14.0 Beta (23A5257q) Aerial still does not function.

Dinesh that you with all the apple monitors?

It is a lot of real estate LOL :)

[DSBlackHeart]

Something I noticed:

I have found when Aerial tries to run, and it did this also under Ventura BTW

when Aerial starts my system fan and processor spin up, it sounds as if my I-Mac is going to take off like Air Force One or something, the fan speed maxes out, and it does this for 20 to 30 minutes.

I reported this but was given the canned response of "Rest The SMC" La Di Da by Apple

So before doing all of that I just turn Aerial and the other "Magic Window" off and uninstalled, does not work anyway for now.

Wow, I have not heard this much quite from my Imac in months, RPM is in the 1200 - 1500 range Normal before it would spin at 2800+

Did anyone else notice? Just curious? I have been using Macs Fans Control as a monitor for a long time.

[glouel]

Something I noticed:

In the future, please keep the discussion to the topic at hand as it's super hard for me and others who want to follow this precise issue if we talk about other stuff. I created another issue to discuss yours here https://github.com/JohnCoates/Aerial/issues/1310 and will reply to you there. Thanks !

[DSBlackHeart]

Thanks for doing that,

[glouel]

• iScreensaver handles screensaver Options using a helper app. This launches OK, but now in Sonoma, if you hide the Helper app window, and then click the screensaver 'Options' button a second time, the .saver bundle tries to push the helper app frontmost using this code:

           let apps = NSRunningApplication.runningApplications(withBundleIdentifier: childBundleID)
            if let helperApp = apps.first {
                helperApp.activate(options: [ .activateIgnoringOtherApps ])
           }

This is no longer working, but I'm not sure why.

Ok took me a bit to get to you but could it be the new sandboxed by default that denies you this ? It may be fine to launch, but not access the list of running apps in sandbox, for example ? That whole sandbox by default is just an epically unthought through disaster 😩

Companion gets the same prompt for daring to open my own screen saver plugin ! If only there was something like, hmm, an app extension system that would solve this 😅

Edit : See this session about the changes in sandboxing by default : https://developer.apple.com/wwdc23/10053?time=1066 The workaround is granting Full Disk Access, or individually making security bookmarks for external locations. Obviously, screen saver being on the fringe, there are many pitfalls for us (your app can install a screen saver without prompting apparently, which is nice, but you can't open the file or access the container of your screen saver from your app).

[glouel]

Extra notes about Sonoma implementation of Wallpaper/Screen Savers :

• We can find a (new) WallpaperAgent.app that contains extensions for the various fixed/dynamic wallpapers /System/Library/CoreServices/WallpaperAgent.app/Contents/Extensions/, those files are here in Sonoma:

WallpaperAerialsExtension.appex     
WallpaperCAPackageExtension.appex   
WallpaperDynamicExtension.appex     
WallpaperGradientExtension.appex    
WallpaperImageExtension.appex       
WallpaperMontereyExtension.appex
WallpaperVenturaExtension.appex
wallpaper.appextensionpoint

As mentioned, this is all new, and all the wallpapers are .appex now basically. Went back to macOS 13.2 and nothing of the sort there.

• If I set one of the new aerial wallpapers as a wallpaper + screensaver, I can't see any legacyScreenSaver.appex activity. Unlike what I previously assumed, they may not have implemented this as part of legacyScreenSaver.appex, but possibly as part of loginwindow+ something.

• Some interesting events that show some "coordination" between loginwindow and WallpaperAgent as logged by Console.app just after my login was validated and before the screensaver exited :

15:45:18.295753+0200    loginwindow 7E87C2E5: BEGIN - Wallpaper Timeline: XPC Encode
15:45:18.295782+0200    loginwindow 665D8E61: BEGIN - Wallpaper Timeline: XPC JSON Encode
15:45:18.295846+0200    loginwindow 665D8E61: 🧸 - LoggedActivity 'Wallpaper Timeline: XPC JSON Encode' did not call end/error!
15:45:18.295866+0200    loginwindow 7E87C2E5: END - Wallpaper Timeline: XPC Encode
15:45:18.296943+0200    WallpaperAgent  958F9198: BEGIN - Wallpaper Timeline: XPC Decode
15:45:18.297121+0200    WallpaperAgent  EE7B0511: BEGIN - Wallpaper Timeline: XPC JSON Decode
15:45:18.297386+0200    WallpaperAgent  EE7B0511: END - Wallpaper Timeline: XPC JSON Decode

Now I want to see that JSON ! Convertly, I see the invert talking at start :

15:59:25.485346+0200    WallpaperAgent  692CBD1F: BEGIN - Wallpaper Timeline: XPC Encode
5:59:25.486282+0200 loginwindow 964C6BBB: END - Wallpaper Timeline: XPC Decode

So there's some XPC coordination, possibly about video position between wallpaper/loginwindow ?

• Later on, I think this is really interesting :

15:45:18.300165+0200    WallpaperAgent  5D219CD5: END - [com.apple.wallpaper.extension.dynamic] wallpaper(CADCF9A4) set presentation mode to idle

This is the "slowdown" effect, is the wallpaper is a paused avplayer in the case of the WallpaperAerialExtension ? Or do they replace it with a fixed frame grab ?

• So what plays the video during the screensaver ? I'm confused, I see this that makes it look a lot like WallpaperAerialsExtension is the one doing it

15:59:25.697273+0200    WallpaperAerialsExtension   <<<< VMC >>>> vmc2GMFigLogDumpStats: VMC(0x10f014600): client pid 12733; timebase time: 39.940 s rate: 0.00. recent stats: cfq empty: 46 times; frames decoded: 14, dropped: 0; decode: mean 0.000 s, max: 0.000 s; codec service tier: 0.
5:59:25.700139+0200 WallpaperAerialsExtension   [<private>]: minUpcomingOriginalPTS.seconds 39.939940 - self.firstPrerolledPTS!.seconds 39.939940 = 0.000000
15:59:25.704030+0200    WallpaperAerialsExtension   [<private>]: minUpcomingOriginalPTS.seconds 40.197940 - self.firstPrerolledPTS!.seconds 39.939940 = 0.258000
15:59:25.704063+0200    WallpaperAerialsExtension   [<private>]: Start timebase at 39.939940
15:59:25.704408+0200    WallpaperAerialsExtension   <<<< LayerSync >>>> figlayersync_setLayerTiming: (layer 0x600001467bc0) set layer speed: 1.000000
15:59:25.704514+0200    WallpaperAerialsExtension   <<<< FigVideoQueue >>>> vq_timebaseEffectiveRateChanged: (0x11d909ed0) Effective Timebase Rate of videoqueue changed to: 1.000 (pid 12733)

So basically :

• the video is stopped (rate: 0)
• the video moves to some point in time (start timebase)
• the video is started (rate: 1)

If I had to guess how this all works :

• when you login, the video plays
• after login it stops playing at a given position, there's some XPC encoding that send that position to WallpaperAgent from loginwindow
• a fixed frame grab is generated and used as the wallpaper to preserve resources
• when you launch the screensaver, some XPC encoding from WallpaperAgent sends the time position of the current screenshot to loginwindow. loginwindow starts WallpaperAerialsExtension (???) at that position so it can animate the video. Playback starts
• when you stop, loginwindow sends back the stop position to WallpaperAgent who will make a frame grab of the video again (maybe running or they are sharing WallpaperAerialsExtension ?? this is so weird).

I'll say this, I hope I've misunderstood this, I thought they would literally share the view between loginwindow/desktop/screensaver, and not just pass stuff around, try to coordinate video players, etc. This is certainly resource intensive (and basically was my first thought about how I would have implemented this between Aerial and Companion because I would have had no other choice). Weird.

• Does this help anything ? Well outside the curiosity of how they did it, one thing is clear, at no point is legacyScreenSaver implicated. So why then, when we do set our savers do we see those legacyScreenSaver (Wallpaper) running ? Is it just loginwindow interacting with legacyScreenSaver as if it was WallpaperAerialExtensions ? Did Apple really intend for legacyScreenSaver to be ran as a wallpaper ? Who knows !

Edit : regarding the legacyScreenSaver (Wallpaper) we've all seen in Activity Monitor, if you use Ventura as a screen saver (which is an .appex), you end up with Ventura (Wallpaper) too. So I think that confirms they relaunch any appex as a wallpaper (perhaps mistakenly, or maybe they'll give it to use some day too!).

[glouel]

Also, something - for developers - that took me a while to understand : if legacyScreenSaver is still running in the background a version of your screen saver, and you install a new version, the new version will show up in System Settings... but not when running under ScreenSaverEngine.

Force quitting legacyScreenSaver (Wallpaper) will fix the issue and let you try your new build.

Edit : This also goes when updating Aerial for users.

[glouel]

Ok so, again trying to understand how to workaround the beta 2 situation, I kept wondering why Aerial didn't stop and noticed two things :

• stopAnimation is never called on us (I want to say this is not particularly new, but usually we get killed)
• the useful event I used as a workaround to pause video playback and patiently wait to be killed, com.apple.screensaver.willstop, is not sent when the screensaver is supposed to stop

After tracking down those events in Console, I noticed this :

par défaut  15:16:25.082087+0200    distnoted   register name: com.apple.screensaver.willstart object: kCFNotificationAnyObject token: 3f00000037 pid: 47665
par défaut  15:16:25.082693+0200    distnoted   register name: com.apple.screensaver.willstop object: kCFNotificationAnyObject token: 4000000036 pid: 47665

So those events (willstart and willstop) are just fired at the same time, which is pretty terrible for my workaround and Companion desktop mode (where I monitor those events to stop/start playback while the screensaver runs).

This is filed as FB12523305

Workaround, I found none at the time, I can't see any event firing up that I could use at the moment to correctly detect this situation but will keep looking (I do see a bunch of events fired every 10 seconds though!).

As a corollary to this, it keeps making the picture clearer, from our point of view, we just get never stopped, and maybe maybe, the appex may be "passed around" from ScreenSaverEngine to Wallpaper process, explaining what we see in Activity Monitor.

Edit : more stuff, per my previous post, legacyScreenSaver keeps us in memory, but this is how it tries to do it. In this scenario, I launcehd the screensaver once (two monitors), exited, launched again :

Here's what Aerial sees :

2023-07-05 15:49:16.208 : 🖼️ <AerialView: 0x13af2ace0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-05 15:49:16.209 : 🖼️ <AerialView: 0x13afe4a90> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)

note the views, acec0, 4a90, those are the "previously launched and kept around" views, they do receive a new startAnimation event each.

Yet... I see two new launched

2023-07-05 15:49:16.360 : 🖼️ AVinit (.saver) (0.0, 0.0, 2560.0, 1440.0) p: false o: false
2023-07-05 15:49:16.361 : 🖼️ <AerialView: 0x11d21d3c0> AV setup init (V3.2.7beta2) preview: false
2023-07-05 15:49:16.469 : 🖼️ AVinit (.saver) (0.0, 0.0, 2560.0, 1440.0) p: false o: false
2023-07-05 15:49:16.469 : 🖼️ <AerialView: 0x13ba28460> AV setup init (V3.2.7beta2) preview: false

So, are we just getting stacked and laying around running ? The answer seems to be yes, a few minutes later :

2023-07-05 15:54:07.407 : <AerialView: 0x13af2ace0> played did reach end

So ace0 is still here, playing in the background, never stopped. Sigh.

I can definitely confirm this with some new logging I added earlier, where each AerialView can register to listen to audio events :

2023-07-05 15:56:29.570 : 🎧 media remote callback
2023-07-05 15:56:29.574 : 🎧 audio info
2023-07-05 15:56:29.575 : 🎧 The Veronicas - Untouched (Hook Me Up (Bonus Track Version)) with artwork 
2023-07-05 15:56:29.575 : 🎧🟧 updateStatus
2023-07-05 15:56:29.575 : 🎧🟧 updateStatus
2023-07-05 15:56:29.575 : 🎧🟧 updateStatus
2023-07-05 15:56:29.576 : 🎧🟧 updateStatus
2023-07-05 15:56:29.576 : 🎧🟧 updateStatus
2023-07-05 15:56:29.576 : 🎧🟧 updateStatus

Note the 6 status updates, after launching the screensaver 3 times (two monitors)

And after a bunch more launches :

2023-07-05 15:59:09.605 : 🎧 media remote callback
2023-07-05 15:59:09.616 : 🎧 audio info
2023-07-05 15:59:09.617 : 🎧 Louisadonna - À deux (Fatigue) with artwork 
2023-07-05 15:59:09.617 : 🎧🟧 updateStatus
2023-07-05 15:59:09.617 : 🎧🟧 updateStatus
2023-07-05 15:59:09.617 : 🎧🟧 updateStatus
2023-07-05 15:59:09.618 : 🎧🟧 updateStatus
2023-07-05 15:59:09.618 : 🎧🟧 updateStatus
2023-07-05 15:59:09.618 : 🎧🟧 updateStatus
2023-07-05 15:59:09.618 : 🎧🟧 updateStatus
2023-07-05 15:59:09.618 : 🎧🟧 updateStatus
2023-07-05 15:59:09.619 : 🎧🟧 updateStatus
2023-07-05 15:59:09.619 : 🎧🟧 updateStatus
2023-07-05 15:59:09.619 : 🎧🟧 updateStatus
2023-07-05 15:59:09.619 : 🎧🟧 updateStatus
2023-07-05 15:59:09.619 : 🎧🟧 updateStatus
2023-07-05 15:59:09.619 : 🎧🟧 updateStatus

That's a lot of AerialViews running! If you use Sonoma beta2, if you see things slowing down, open Activity Monitor, filter for "wallpaper" and kill legacyScreenSaver (Wallpaper) or others. Surprisingly the CPU usage isn't that bad for Aerial. Ventura screensaver does way worse in that scenario.

Last edit : Maybe it's me or I just get lucky, but I have a feeling that after force quitting legacyScreenSaver, we get instantiated on the correct monitors (with caveats).

[glouel]

Beta3 is out, from a quick glance I see no change :

• from a user perspective, it's still bugged for multimonitors when applied on all screens (you get one, the other, or both)
• willstop event is still sent at launch just after willstart
• we hang around as legacyScreenSaver (Wallpaper) and our views are never stopped or killed, they keep running in the background

As a user, I would recommend :

• Setting Aerial on only one screen, set an apple screen saver on other monitors for the time being. At least this should ensure "proper" functionnality
• Periodically kill legacyScreenSaver (Wallpaper) (and all other stuff with (Wallpaper) in the name, any screen saver you launch will be kept around, this is not just Aerial or 3rd party, this is all screen savers, including Apple's one)

[glouel]

New bug, manually launching ScreenSaverEngine used to properly launch the screen saver (from this link : https://stackoverflow.com/questions/4505198/how-can-i-start-the-screensaver-and-lock-the-screen-from-the-os-x-terminal ). This no longer works now, you get either unable to reach the lock screen or get in a lock screen loop. This affects Companion launch screen saver button, beta8 is incoming with a fix (which is to use a private API).

Reported as FB12542012

[DSBlackHeart]

Thank You for all the updates, Looking forward to when this works again.

(Beginning to wonder if I need to break out the Flying Toasters) Shows how old I am LOL

[glouel]

Ok so, slightly good news, I found a "kinda" workaround to our CPU woes under Sonoma. It's not a perfect fix but basically, instead of waiting for com.apple.screensaver.willstop, I now use com.apple.screenIsUnlocked to pause the video playback, which massively reduce background CPU usage.

This works even if you put a time exemption to your lock screen (say, don't ask password before 5 mins).

This trick will only work with screensavers that implement it, so if you use the Ventura saver alongside Aerial, the Ventura saver will still have major CPU usage (and it stacks every time you launch your screensavers).

This will be available in a new beta of Aerial today. Note: this also fix the pause/resume for Companion integration of the video wallpaper.

[glouel]

It's out here : https://github.com/JohnCoates/Aerial/releases/tag/v3.2.7beta3

If you install, make sure you kill legacyScreenSaver (Wallpaper) first.

Of note:

• it's a workaround and not a great solution. The event is triggered technically before the screensaver is hidden so you will see Aerial pause during the transition to your desktop, it's not very smooth but it's better than high cpu usage for now
• new beta3 doesn't seem to change anything from a quick look, multi monitor, this issue, I haven't seen much change if any
• if you use companion desktop mode, it will now pause and resume correctly when you launch/stop the screensaver. But it will resume at full speed and not your chosen playback speed. This will be fixed in next Aerial beta

[glouel]

So beta 4 is here and a few early thoughts :

• Multi monitor may be fixed ? At least for two monitors ?
• Sometimes, screensavers just don't launch ? This seems to happen if I try to launch multiple times in a row quickly via a hot corner. It starts working again if I wait a few seconds.
• legacyScreenSaver (Wallpaper) is still a thing, meaning, we are still running in the background. But Wallpaper is not localised (eg here in french) :
• System Settings showing the wrong preview, still a thing
• I'm seeing this colorful things in my logs, which may mean that the com.apple.screensaver.willstop are now correctly fired when supposed to ? This needs double checking :

2023-07-25 20:08:38.718 : 🖼️ 📢📢📢 🖼️ 📢📢📢 ☢️sonoma☢️ workaround IGNORING willStop
2023-07-25 20:08:38.720 : 🖼️ 📢📢📢 🖼️ 📢📢📢 ☢️sonoma☢️ workaround IGNORING willStop
2023-07-25 20:08:38.749 : 🖼️ 📢📢📢 ☢️sonoma☢️ workaround screenIsUnlocked
2023-07-25 20:08:38.750 : 🖼️ 📢📢📢 ☢️sonoma☢️ workaround screenIsUnlocked

• Just to clarify, our instances are still stacked on top of each other, so use Aerial betas for the workarounds for that.

I want to say progress. If you use 2+ monitors and see changes please let me know what you get.

[xmddmx]

Testing beta 4 with iScreensaver, I'm still seeing the bug where the screens are launched out of order: screen[1] before screen[0], and the coordinates are wrong (both screens claim to be located at 0,0 )

1131    default 11:43:22.400861-0700    iScreensaver    ########## iScreensaver.init frame=(0.0, 0.0, 1680.0, 1050.0) isPreview=false
1131    default 11:43:22.425282-0700    iScreensaver    ########## iScreensaver.init frame=(0.0, 0.0, 1440.0, 900.0) isPreview=false

When I test with Aerial, this doesn't seem to be a problem, and I am getting 2 monitor playback, which is an improvement. But as you say, there are still leftover legacyScreenSaver (Wallpaper) processes which need to be killed.

I'm also seeing failures when I try to launch the screensaver too quickly in succession: completely blank on both monitors.

[xmddmx]

I'm also having a heck of a time getting the right screensaver to run at all. I can build a debug screensaver using iScreensaver, install it, and it shows up in the Preview mode (and the log confirms the correct bundle is being run). But then when I try to run the screensaver, the Ventura screensaver runs instead.

However, if I go back to an earlier build, and select it in the Screen Savers window, and both Preview mode and full screen Hotcorner work normally (at least the correct screensaver runs, though it's buggy).

Are you having any trouble having more than one copy of Aerial installed and finding the OS won't launch the correct one?