Animate window move/resize/swap/warp

[dominiklohmann]

Discussion / Feature request

This is a long shot, and probably far off in the future. This has been previously discussed in https://github.com/koekeishiya/chunkwm/issues/144.

I've dug up some stuff regarding this.

• How Chromium resizes and moves windows using CGS* functions https://github.com/ChromiumWebApps/chromium/blob/c7361d39be8abd1574e6ce8957c8dbddd4c6ccf7/chrome/browser/ui/cocoa/constrained_window/constrained_window_animation.mm
• A (very old) blog post explaining how CGSSetWindowWarp works http://kevin.sb.org/2006/07/23/cgssetwindowwarp-explained/
• An working example for using CGSSetWindowWarp and CGSSetWindowTranslation for custom animations https://github.com/sailesha/CGSSetWindowWarp-Sample/blob/master/windowAnimation.mm

Obviously all the above examples require access to the NSWindow instance for async animations, which is inaccessible from the Dock process—which is the only process yabai injects code into.

Simply calling SLSSetWindowTransformation with a CGAffineTransform (e..g CGAffineTransformMakeTranslation(100, 100) to move a window 100 right and 100 down) will move the window, but this is not animated. Calling this repeatedly will likely cause lag.

And now here's the long shot that I'd love to see investigated in the future: There are additional functions called SLSTransactionCreate and SLSTransactionCommit, with variants of the transform and warp functions named SLSTransactionWindowTransform and so on. Assuming this works similar to CATransaction on iOS, this could then be used to animate multiple windows with a duration and at the same time.

These SLSTransaction* functions are available (on 10.15 dev beta 4 currently):

nm /System/Library/PrivateFrameworks/SkyLight.framework/SkyLight | grep SLSTransaction

``` 0000000000200d1e T _SLSTransactionAddWindowToSpace 0000000000200f3a T _SLSTransactionAddWindowToSpaceAndRemoveFromSpaces 0000000000201f5c T _SLSTransactionBindSurface 0000000000200129 T _SLSTransactionClearMenuBarSystemOverrideAlphas 00000000001ffce4 T _SLSTransactionClearWindowLockedBounds 00000000001ff834 T _SLSTransactionClearWindowSystemLevel 0000000000202cb6 T _SLSTransactionCommit 0000000000202ee0 T _SLSTransactionCommitCoalescing 00000000001fd88b T _SLSTransactionCreate 0000000000201a22 T _SLSTransactionDeferWindowMoveEvents 00000000002013c1 T _SLSTransactionDestroySpace 00000000001fd788 T _SLSTransactionGetTypeID 000000000031b9d0 b _SLSTransactionGetTypeID.once 000000000031b9d8 b _SLSTransactionGetTypeID.typeID 000000000020109f T _SLSTransactionHideSpace 00000000001ffd7d T _SLSTransactionMoveWindowWithGroup 0000000000201457 T _SLSTransactionMoveWindowsToManagedSpace 000000000020205f T _SLSTransactionOrderSurface 00000000001fe097 T _SLSTransactionOrderWindow 00000000001fe196 T _SLSTransactionOrderWindowGroup 0000000000201c0a T _SLSTransactionOverrideAppSleepNotifications 0000000000201eb2 T _SLSTransactionPostBroadcastNotification 0000000000202dd4 t _SLSTransactionPostCommit 0000000000201d2a T _SLSTransactionPostNotificationToConnection 0000000000200dd1 T _SLSTransactionRemoveWindowFromSpace 0000000000200e84 T _SLSTransactionRemoveWindowFromSpaces 000000000020132b T _SLSTransactionResetSpaceMenuBar 00000000001fff5c T _SLSTransactionResetWindow 00000000001fef10 T _SLSTransactionResetWindowSubLevel 0000000000202922 T _SLSTransactionSetBackdropChameleonContribution 000000000020283c T _SLSTransactionSetChameleonUpdatesEnabled 0000000000201ec6 T _SLSTransactionSetClientAdvisory 0000000000201950 T _SLSTransactionSetEventCapture 0000000000201654 T _SLSTransactionSetManagedDisplayCurrentSpace 0000000000201875 T _SLSTransactionSetManagedDisplayIsAnimating 00000000002001a9 T _SLSTransactionSetMenuBarBounds 00000000001ffff5 T _SLSTransactionSetMenuBarSystemOverrideAlpha 0000000000200a84 T _SLSTransactionSetSpaceAbsoluteLevel 0000000000200bea T _SLSTransactionSetSpaceAlpha 0000000000200b37 T _SLSTransactionSetSpaceOrderingWeight 0000000000200957 T _SLSTransactionSetSpaceShape 000000000020054f T _SLSTransactionSetSpaceTransform 0000000000202175 T _SLSTransactionSetSurfaceBounds 0000000000202563 T _SLSTransactionSetSurfaceOpacity 0000000000202666 T _SLSTransactionSetSurfaceResolution 00000000001fe295 T _SLSTransactionSetWindowAlpha 00000000001fe3c6 T _SLSTransactionSetWindowBrightness 00000000001ff8cd T _SLSTransactionSetWindowGlobalClipShape 00000000001fe4f7 T _SLSTransactionSetWindowLevel 00000000001ff9fd T _SLSTransactionSetWindowLockedBounds 00000000001fe66b T _SLSTransactionSetWindowOpaqueShape 00000000001fe79b T _SLSTransactionSetWindowProperty 00000000001fefa9 T _SLSTransactionSetWindowReleasesBackingOnOrderOut 00000000001feace T _SLSTransactionSetWindowResolution 00000000001fed2a T _SLSTransactionSetWindowShape 00000000001fee5a T _SLSTransactionSetWindowSubLevel 00000000001ff08f T _SLSTransactionSetWindowSystemAlpha 00000000001ff77e T _SLSTransactionSetWindowSystemLevel 00000000001ff1c0 T _SLSTransactionSetWindowTransform 00000000001ff5e9 T _SLSTransactionSetWindowWarp 0000000000201009 T _SLSTransactionShowSpace 000000000020158b T _SLSTransactionSpaceTileMoveToSpaceAtIndex 0000000000201af2 T _SLSTransactionUpdateRegion 000000000020120c T _SLSTransactionWillSwitchSpaces 00000000001fe615 t __SLSTransactionCommitAction 00000000001fdfe4 t __SLSTransactionDecode_CFString 0000000000052ef0 t __SLSTransactionDecode_CGSSpaceIDArray 00000000001fdc6b t __SLSTransactionDecode_packed64 00000000001fdc9e t __SLSTransactionDecode_packed64_tag 0000000000201135 t __SLSTransactionEncode_CFArray 00000000001fdd41 t __SLSTransactionEncode_CFString 00000000002016fd t __SLSTransactionEncode_UUIDString 00000000001fdba3 t __SLSTransactionEncode_packed64 00000000001fd994 t __SLSTransactionEncode_resize 00000000001fd7cd t __SLSTransactionFinalize 00000000001fd925 t __SLSTransactionPopWriteStream 0000000000201d3a t __SLSTransactionPostNotification 0000000000202a53 t __SLSTransactionPushWriteStream 0000000000202bf6 t __SLSTransactionResetWriteStream 000000000031b9e8 b __SLSTransactionWriteStreamKey.key 000000000031b9e0 b __SLSTransactionWriteStreamKey.once 000000000020300f t ___SLSTransactionCommitCoalescing_block_invoke 00000000001fd7b4 t ___SLSTransactionGetTypeID_block_invoke 0000000000201507 t ___SLSTransactionMoveWindowsToManagedSpace_block_invoke 0000000000202ed8 t ___SLSTransactionPerformCommitAction 0000000000202520 t ___SLSTransactionSetSurfaceBounds_block_invoke 0000000000202808 t ___SLSTransactionSetSurfaceResolution_block_invoke 00000000001fe5ef t ___SLSTransactionSetWindowLevel_block_invoke 00000000001feca3 t ___SLSTransactionSetWindowResolution_block_invoke 00000000002012a5 t ___SLSTransactionWillSwitchSpaces_block_invoke 00000000001fdad3 t ____SLSTransactionWriteStreamKey_block_invoke ```

Why? Look at this example from i3. Looks nice, doesn't it?

[FelixKratz]

An unexpected behavior is:

• When Screen Capture Permission is not given, borders will frequently be ordered above the window.

[wxwern]

Not sure if these are related to whatever that's mentioned here or if I have any configuration issues:

• A topmost floating window's animation proxy doesn't seem to be topmost, so the proxy might go underneath windows and be entirely invisible while a move animation is in progress. To reproduce: with yabai -m config window_topmost on and two bsp windows, call yabai -m window --toggle float --grid 8:8:1:1:6:6 on one of them.

• It might look better if the animation proxy (for whichever window is in focus after the animation) is on top of the same layer throughout the animation. Example: yabai -m window --focus stack.last --stack east --focus stack.last should have the eastern window stay on top while it animates onto the stack.

• There's (sometimes) a flicker where borders will reappear at the previous location at the end of an animation.

• Hiding and unhiding an app that has windows across multiple spaces causes animation proxies for all spaces to be created and be visible even though the spaces themselves aren't visible.

• Unhiding an app that has windows may cause it to appear to flicker between spaces with the app's windows (sometimes indefinitely and I'd need to restart yabai to resolve that).

• Insert feedback borders no longer match the window border properties (like width) specified in config.

PS: edited to add additional issues I've seen so far

On a side note, it'd be nice if we can optionally disable the background blur from the borders if we prefer not to use it or if we aren't making windows transparent.

Thanks for all the work on this!

[koekeishiya]

• A topmost floating window's animation proxy doesn't seem to be topmost, so the proxy might go underneath windows and be entirely invisible while a move animation is in progress. To reproduce: with yabai -m config window_topmost on and two bsp windows, call yabai -m window --toggle float --grid 8:8:1:1:6:6 on one of them.

• There's (sometimes) a flicker where borders will reappear at the previous location at the end of an animation.

• Hiding and unhiding an app that has windows across multiple spaces causes animation proxies for all spaces to be created and be visible even though the spaces themselves aren't visible.

• Unhiding an app that has windows may cause it to appear to flicker between spaces with the app's windows (sometimes indefinitely and I'd need to restart yabai to resolve that).

These should all be fixed in the latest commit.

[koekeishiya]

Added the ability to configure border HiDPI mode (on/off) and border blur (on/off). Also updated the docs to specify which commands need SIP to be partially disabled -- should make it easier than having to check the wiki listing.

window_border_hidpi [<BOOL_SEL>]
    Draw border in high resolution mode; for High Dots Per Inch ("Retina") displays.
    HiDPI uses significantly more memory.

window_border_blur [<BOOL_SEL>]
    Blur border allowing it to act as a backdrop for transparent windows.

[koekeishiya]

Going to merge into master and do whatever remaining changes are required there.

[koekeishiya]

Insert feedback borders no longer match the window border properties (like width) specified in config.

Fixed in latest commit. Insert feedback borders now use both the width and radius setting properly.

[wxwern]

• I read that this should be fixed but I'm not sure why I'm still seeing some border flicker here

  

• It also appears that the animations disappear completely for me if i warp a window A towards B in this state (i.e. focus window A, then yabai -m window --warp east):

  +-----+-----+
  |     |     |
  +-----+  B  +
  |  A  |     |
  +-----+-----+

  which leaves a period of time where all windows are invisible. EDIT: This occurs only when auto_balance is set to on.

I'm currently on the latest commit on the master branch.

[koekeishiya]

It also appears that the animations disappear completely for me if i warp a window A towards B in this state (i.e. focus window A, then yabai -m window --warp east):

I can reproduce this with auto_balance on.

I read that this should be fixed but I'm not sure why I'm still seeing some border flicker here

I cannot reproduce this on either of my machines (older Intel x64 running Big Sur and Apple Silicon running Monterey). Not really sure how that would be happening, as resizing happens while the proxy window is showing instead of the window, and then a transactional swap is performed at the end.

[wxwern]

I cannot reproduce this on either of my machines (older Intel x64 running Big Sur and Apple Silicon running Monterey). Not really sure how that would be happening, as resizing happens while the proxy window is showing instead of the window, and then a transactional swap is performed at the end.

Update: It seems like the flicker I encountered isn't linked to the end of the animation at all. It also seemed like having yabai signals might be worsening/delaying the effect and makes the overall operation slower.

If I slow down the animation duration further, and remove all window_moved and window_resized signals I hooked up, the flicker looks something like this:

The config I used to reproduce the gif above: (colours and padding set for visibility)

  yabai -m config \
      top_padding 4 \
      bottom_padding 4 \
      left_padding 4 \
      right_padding 4 \
      window_gap 4 \
      active_window_border_color 0xffffffff \
      normal_window_border_color 0xff555555 \
      window_border on \
      window_border_width 2 \
      window_animation_duration 1.0 \
      window_topmost on \
      layout bsp

have two windows in bsp, then run yabai -m window --toggle float --grid 8:8:1:1:6:6 on one of the windows.

[koekeishiya]

Right, ,I understand why it is happening. I'll have to rethink parts of the system for other reasons too; currently it messes with window ordering in a way that causes problems when combining commands, e.g having multiple windows of the same application on space 1, and then doing yabai -m window --space 3 && yabai -m space --focus 3, when we restore windows at the end of the animation, that window steals focus (when we tell macOS to make it visible on screen). This only happens if there is a window of the same application on that space.

[koekeishiya]

@wernjie Do you still have this issue on the latest master? I can't reproduce it with the config you posted.

[koekeishiya]

I believe all known reported issues should be resolved now on master. Please let me know if you still encounter issues.

It might look better if the animation proxy (for whichever window is in focus after the animation) is on top of the same layer throughout the animation.

I think it would become very complicated to try and catch a focused window event and have that impact the state of a live animation.

[wxwern]

Do you still have this issue on the latest master? I can't reproduce it with the config you posted.

It seems like the flicker is much improved on the latest master with that config (~9 out of 10 times it no longer shows up). Though, I can still see a flicker frequently when I have window move/resize signals added which update my Übersicht status bar. Not sure why it only appears sometimes.

For now I can make do with the lack of those signals. Do let me know if there's anything else I can provide to help debug the issue.

[koekeishiya]

I was able to catch it in a single frame, at the end of an animation, on my screen.

Edit: The issue is that we let the border position/size update through the window notification callbacks. These go through the event queue and needs to be processed one at a time. If there are a lot of requests to yabai in the timespan while the animation is running, and the animation duration is fairly short, we might end up in a situation where the animation finishes and the border is made visible (proxy is destroyed), before the border has been able to move/reposition through the event callback.

A solution would be to set the border frame immediately when we set the frame of the window it belongs to, however that will require us to do another call to the AX API to retrieve the frame to use, because of possible min/max size constraints. This is probably fine and won't make a noticeable difference in terms of performance.

[wxwern]

That makes sense. The latest commit has also fixed the flicker for me. Thanks!

[koekeishiya]

If no new bugs are detected regarding this issue, I plan to do a release within the weekend.

[koekeishiya]

Meh think I'll just gamble; I haven't discovered any bugs so far during my normal usage. If there is a bug, simply setting window_animation_duration 0.0 will make it work like v4.0.4 so should be fine.

[dominiklohmann]

Found a bug with the animations: With SIP enabled (and thus no scripting addition loaded), and window_animation_duration set to a non-zero value, there sometimes is a very wonky animation happening:

https://user-images.githubusercontent.com/4488655/192065451-171f7dc0-ba23-43d4-8113-40e3c3f23499.mp4

Note how in the capture I needed to swap the window three times to get the weird animation. I did not find a way to make this deterministic yet.

Thanks a lot for implementing this, with the scripting addition loaded it works really well. Much better than I imagined possible all that time ago when I opened this issue.

[ubuntudroid]

That's an awesome feature! Thanks a lot! ❤️

However, I also have problems with SIP enabled: the animations are only playing very rarely when swapping or moving windows. Only about every 7th (or even more) command triggers an animation. Not predictably though.

yabai -m config window_animation_duration    0.35

UPDATE Actually it looks like that's actually the same thing @dominiklohmann reported, just that our assumptions on how we think it should work differ. 😏 I thought every swap/move etc. should trigger an animation. But maybe I'm wrong here.

[koekeishiya]

Found a bug with the animations: With SIP enabled (and thus no scripting addition loaded), and window_animation_duration set to a non-zero value, there sometimes is a very wonky animation happening:

So this is happening if you perform some action that would cancel an in-progress animation for a window and start a new animation towards a new state. If you set the animation duration to 1.0 seconds, and do yabai -m window --swap west followed by yabai -m window --swap east shortly after, you will trigger that behaviour.

What's happening is that the first swap command will trigger the start of an animation, however we need the scripting addition to hide the real window while we animate a proxy; performing a transactional switch so that it isn't actually obvious to the user that this is what's happening. Because the scripting-addition is not loaded, the swap doesn't occur, and there is no animation showing.

When an animation is cancelled because a new one is started, the proxy is updated and swaps with the old proxy. This swap does not rely on the scripting addition, which is why those windows suddenly become visible.

I do believe I stated that this particular config option requires SIP to be disabled. However, I agree that maybe there should be some limitation in place preventing the value to be changed from 0.0 on systems where SIP is enabled.

There is no way to allow this functionality with SIP enabled, unfortunately.

[ubuntudroid]

I do believe I stated that this particular config option requires SIP to be disabled. However, I agree that maybe there should be some limitation in place preventing the value to be changed from 0.0 on systems where SIP is enabled.

There is no way to allow this functionality with SIP enabled, unfortunately.

That makes sense. Think I should have read the docs first - just saw it in the changelog and jumped right into it. 😅 My bad!

[koekeishiya]

Added a guard that only allows window_animation_duration to be set if the appropriate SIP flags are disabled.

[FelixKratz]

The animation can get fairly stuttery at times. My testing shows that this is connected to disabling the update on border resize and border redraw. Even though animations use a different SLS connection, they seem to affect each other still.

This patch https://github.com/FelixKratz/yabai/commit/a20e36e017e4e3257b9904625a6b0f080d9e1136 makes the animations (especially noticeable on large displays and short animation durations) dramatically smoother for my configuration. It however introduces redraw flickering since the update has not been disabled on redraw.

[jakenvac]

Is it normal for there to be a delay before the animations play? I'm using the latest version on macOS 12.6 apple silicon.

After using yabai -m window --warp west there's about a 500ms delay before the windows will actually swap.

[koekeishiya]

Is it normal for there to be a delay before the animations play?

This delay is caused by actually setting the change in size/position using the accessibility API, and there is nothing that can be done to speed up that operation. The accessibility API is janky and that's really it - until Apple either improves the API/performance of it, or implements a different system for third-party software to interact with windows/applications.

[albert-ying]

can window open and close animations be added?

It might be possible to create a window opening animation; depends on how much delay there is between the window spawning and yabai getting notified, I guess this would likely be implemented as a fade+scale transition.

Animating on window close is pretty much impossible. The window is already inaccessible from system APIs by the time we are notified that a window has been destroyed.

What if we let the yabai to send the window closing signal? Say implement a command like yabai -m close_current_window and assign cmd+w to it using shkd

[koekeishiya]

What if we let the yabai to send the window closing signal? Say implement a command like yabai -m close_current_window and assign cmd+w to it using shkd

That would probably work, but I'd prefer something more universal. I often use cmd+q instead of cmd+w if it is the last window of an application, and that would not trigger the animation with this solution.

Not that I am fully opposed to experimenting with something like what you mentioned.

[FelixKratz]

I have pondered a bit further with the animation system and came up with a new POC for a fade between the scaled window and the unscaled window at the end of an animation. I believe this makes animations a lot more smooth https://github.com/FelixKratz/yabai/commit/de13dbca8d62a8a453d15455c7f3adeb527a8e09:

https://user-images.githubusercontent.com/22680421/209452263-2432c9a0-6e17-4774-92c4-0404ddad9c2d.mp4

[koekeishiya]

How does that look when you hit windows that have min/max size constraints?

[FelixKratz]

How does that look when you hit windows that have min/max size constraints?

A bit funny:

https://user-images.githubusercontent.com/22680421/209884625-dedef2b4-9253-40b9-8478-1ef260e5c97e.mp4

[PrayagS]

I have pondered a bit further with the animation system and came up with a new POC for a fade between the scaled window and the unscaled window at the end of an animation. I believe this makes animations a lot more smooth FelixKratz@de13dbc:

@FelixKratz I'm on the latest release v5.0.2 and enabled animation but noticed a different behavior on my terminal windows.

The text resizes after the window is set to its new location. In your video, the text resizing and window movement animation happen simultaneously. What could be the reason for this? Does your branch still have a different code?

I'm using the kitty terminal. And frame rate is set to 120 in yabai.

[koekeishiya]

Is it normal for there to be a delay before the animations play?

. This delay is caused by actually setting the change in size/position using the accessibility API, and there is nothing that can be done to speed up that operation. The accessibility API is janky and that's really it - until Apple either improves the API/performance of it, or implements a different system for third-party software to interact with windows/applications.

This started to annoy me and I got a little bit creative, and found a way to improve the situation.

https://github.com/koekeishiya/yabai/issues/2060

[net]

Would love a way to just disable animations for any resize.

[koekeishiya]

window_animation_duration 0.0 ?

[net]

I mean disable only for resize operations, but not for move operations. So any animations that are compromised by the stretching effect, while keeping the animations that look perfect.

[koekeishiya]

I'm not sure if that level of complexity can be handled reasonably in the animation system. I agree that the abrupt end from final animated state to displaying the actual window is not ideal.

[koekeishiya]

@net https://github.com/koekeishiya/yabai/issues/2137

Please try the latest master; is it better or worse in your opinion? Make sure to uninstall the scripting addition and reinstall it after compiling.

Edit: I think it is a lot nicer actually.

[net]

I think it's better, but still not something I'd leave enabled. The stretching itself is the issue. I just noticed the stretching causes window corners to deform too when their ratio changes.

The only way I can think to do it, given the constraints, in an aesthetically pleasing way—and this way admittedly would take a lot more work—is to apply a strong gaussian blur to the overlay so you can't really see the window content deforming, and mask the corners so they look like standard macOS corners throughout the full animation.

https://github.com/koekeishiya/yabai/assets/6983821/a9720b29-72c7-465b-9ffb-67ee245a822c

[net]

Actually, here's another aesthetically pleasing option, this one even more complicated, but perhaps with the best possible effect:

1. For resizes narrowing a window, do not deform the window image, but rather animate the width down by masking in from the right side.
2. For resizes shortening a window, do the same as above, but masking up from the bottom.
3. For resizes widening a window, take the color of a pixel on the rightmost edge of the window image, preferable near the top just before the corner radius of top right corner begins (to increase the chance of capturing the window's UI background color). Use that color as a background fill color to animate a mask out from the right side of the window image, revealing the background fill as it animates.
4. For resizes lengthening a window, do the same as above, but masking down from the bottom.

A resize may use various combinations of the above options. After the resize, possibly alpha-fade to the window's true UI. The effect of this should be to appear as if the window doesn't relayout its UI until after the animation, but that the window frame itself still cleanly animates.

[koekeishiya]

I think that is too much work for the actual processor to do for this to feel snappy, with the way this has to work. Preparing animation state already takes between 60-130ms depending on a variety of conditions.

I think we looked at how Apple performs transitions a bit further up this thread, and they fade transition between the old and new state. Admittedly their version looks better as they have access to more info than yabai does.

[koekeishiya]

FelixKratz did a more invasive alpha fade POC a couple of comments above. The window transitions approximately in the middle of the animation, so the stretch is less pronounced. Do you think such a solution would be acceptable? I guess it would be hard to tell without trying it yourself and observe the results directly. Looking at a gif is not always the same/as accurate.

[net]

Specifically, Apple does it by performing the alpha fade during the first two thirds of the animation, with some easing function that performs most of the fade within the first few frames. As you noted, this has the effect that by the time the resize animation is slowing down towards the end, the window has already faded to the new state, and so the warping is much less noticeable. They don't even do any corner masking.

I think using the same timings as Apple is as good as anyone can ask for 😄