Missing animations when minimizing/maximizing/restoring undecorated window

[mahozad]

Describe the bug I want to have custom title bar for my Desktop application (especially important to implement dark theme). So, I set undecorated = true.

With undecorated in Windows OS, when maximizing or minimizing a window or restoring a window already minimized to taskbar, the window does not animate. Decorated windows, in contrast, have proper animations:

Decorated window	Undecorated window

I tried the workaround in #3166 but it does not seem to work for undecorated windows (window minimizes but with no animation):

fun minimize() {
    User32.INSTANCE.ShowWindow(
        WinDef.HWND(Pointer((window as ComposeWindow).windowHandle)),
        User32.SW_MINIMIZE // == 6
    )
    // OR
    User32.INSTANCE.CloseWindow(
        WinDef.HWND(Pointer((window as ComposeWindow).windowHandle))
    )
}

Electron seems to have had this problem too in the past:

• https://github.com/electron/electron/issues/32993
• https://github.com/electron/electron/issues/751

This is the code they use to set proper windows decoration and preserve animations:

https://github.com/electron/electron/blob/3a5e2dd90c099b10679364642a0f7fa398dce875/shell/browser/native_window_views.cc#L367-L390

So, then tried setting just the flags that Electron sets like this for the window:

window(// ...) {
    val frameStyle = User32.WS_OVERLAPPED or User32.WS_MINIMIZEBOX
    User32.INSTANCE.SetWindowLong(
        WinDef.HWND(Pointer(window.windowHandle)),
        User32.GWL_STYLE,
        frameStyle
    )
    App()
}

OR modifying the existing window flags:

window(// ...) {
    val defaultStyle = User32.INSTANCE.GetWindowLong(WinDef.HWND(Pointer(window.windowHandle)), GWL_STYLE)
    val newStyle = defaultStyle and User32.WS_THICKFRAME.inv() and User32.WS_DLGFRAME.inv()
    User32.INSTANCE.SetWindowLong(
       WinDef.HWND(Pointer(window.windowHandle)),
       User32.GWL_STYLE,
       newStyle
    )
    App()
}

The default close, maximize, minimize buttons are removed but cannot get rid of the title bar.

The IntelliJ IDEA 2023.1 is undecorated and has proper animations. Maybe you could consider getting a little help from them?

Fixing this problem could probably resolve or help resolve all the following issues:

• https://github.com/JetBrains/compose-multiplatform/issues/3295
• https://github.com/JetBrains/compose-multiplatform/issues/3164
• https://github.com/JetBrains/compose-multiplatform/issues/2062
• https://github.com/JetBrains/compose-multiplatform/issues/1472

Affected platforms

• Desktop

Versions

• Kotlin version*: 1.8.20
• Compose Multiplatform version*: 1.4.1
• OS version(s)*: Windows 10, Windows 11
• OS architecture (x86 or arm64): x64
• JDK (for desktop issues): Oracle JDK 17.0.5

To Reproduce Compare app minimize animation in these two:

decorated (default title bar):

fun main() {
    application {
        Window(
            state = rememberWindowState(size = DpSize(400.dp, 300.dp)),
            undecorated = false,
            transparent = false,
            onCloseRequest = ::exitApplication,
        ) {
            Text(text = "This is an example text")
        }
    }
}

undecorated (custom title bar):

fun main() {
    application {
        Window(
            state = rememberWindowState(size = DpSize(400.dp, 300.dp)),
            undecorated = true,
            transparent = true, // for rounded corners
            onCloseRequest = ::exitApplication,
        ) {
            Surface(
                modifier = Modifier
                    .fillMaxSize()
                    .border(Dp.Hairline, Color.Gray, RoundedCornerShape(8.dp))
                    .clip(RoundedCornerShape(8.dp))
            ) {
                Column {
                    WindowDraggableArea(modifier = Modifier.fillMaxWidth().height(30.dp)) {
                        Row(
                            horizontalArrangement = Arrangement.SpaceBetween,
                            modifier = Modifier.fillMaxWidth()
                        ) {
                            Text(text = "Title")
                            Row {
                                Box(
                                    contentAlignment = Alignment.Center,
                                    modifier = Modifier
                                        .width(48.dp)
                                        .fillMaxHeight()
                                        .clickable { window.isMinimized = true }
                                ) {
                                    Icon(
                                        imageVector = Icons.Custom.Minimize,
                                        contentDescription = null,
                                        modifier = Modifier.size(12.dp)
                                    )
                                }
                                Box(
                                    contentAlignment = Alignment.Center,
                                    modifier = Modifier
                                        .width(48.dp)
                                        .fillMaxHeight()
                                ) {
                                    Icon(
                                        imageVector = Icons.Custom.Close,
                                        contentDescription = null,
                                        modifier = Modifier.size(14.dp)
                                    )
                                }
                            }
                        }
                    }
                    Text(text = "This is an example text")
                }
            }
        }
    }
}

Expected behavior Undecorated windows should have animations when minimizing/maximizing/restoring them.

[pjBooms]

Relates to https://github.com/JetBrains/compose-multiplatform/issues/3166

[mahozad]

A good example:

• https://github.com/rossy/borderless-window
• https://discourse.glfw.org/t/making-a-custom-titlebar/2392/3

[mahozad]

https://github.com/JetBrains/compose-multiplatform/assets/29678011/e86cb4ba-8233-4cc4-9109-3f8e51a0d1fa

Fixed the problem in Windows (don't know if the problem applies to Linux or macOS). Tried it successfully in Windows 7, Windows 10, Windows 11.

The animation solution was adopted from https://github.com/kalibetre/CustomDecoratedJFrame which was for Swing (Jframe) windows.

With this, native animations for window appearance (open), window disappearance (close), window minimize, window restore, and (possibly) window maximize work correctly. The repo above also implemented Aero snap feature (and possibly, shaking window title to hide other windows) but I omitted it.

It also provides native window shadows and border (and rounded corners in Windows 11).

The only downside is that the window cannot be made transparent (otherwise, it loses animations and shadows).

To be honest, I don't know how the CustomWindowProcedure below works. But made it to work.

Update: 2024-05-07

implementation("net.java.dev.jna:jna-jpms:5.14.0")
implementation("net.java.dev.jna:jna-platform-jpms:5.14.0")
// ...

import com.sun.jna.Native
import com.sun.jna.Pointer
import com.sun.jna.platform.win32.BaseTSD.LONG_PTR
import com.sun.jna.platform.win32.User32
import com.sun.jna.platform.win32.WinDef.*
import com.sun.jna.platform.win32.WinUser.WM_DESTROY
import com.sun.jna.platform.win32.WinUser.WindowProc
import com.sun.jna.win32.W32APIOptions
// ...

fun main() = application {
    Window(
        undecorated = true, // This should be true
        transparent = false,
        // ...
    ) {
        val windowHandle = remember(this.window) {
            val windowPointer = (this.window as? ComposeWindow)
                ?.windowHandle
                ?.let(::Pointer)
                ?: Native.getWindowPointer(this.window)
            HWND(windowPointer)
        }
        remember(windowHandle) { CustomWindowProcedure(windowHandle) }
        MyAppContent()
    }
}

@Suppress("FunctionName")
private interface User32Ex : User32 {
    fun SetWindowLong(hWnd: HWND, nIndex: Int, wndProc: WindowProc): LONG_PTR
    fun SetWindowLongPtr(hWnd: HWND, nIndex: Int, wndProc: WindowProc): LONG_PTR
    fun CallWindowProc(proc: LONG_PTR, hWnd: HWND, uMsg: Int, uParam: WPARAM, lParam: LPARAM): LRESULT
}

// See https://stackoverflow.com/q/62240901
@Structure.FieldOrder(
    "leftBorderWidth",
    "rightBorderWidth",
    "topBorderHeight",
    "bottomBorderHeight"
)
data class WindowMargins(
    @JvmField var leftBorderWidth: Int,
    @JvmField var rightBorderWidth: Int,
    @JvmField var topBorderHeight: Int,
    @JvmField var bottomBorderHeight: Int
) : Structure(), Structure.ByReference

private class CustomWindowProcedure(private val windowHandle: HWND) : WindowProc {
    // See https://learn.microsoft.com/en-us/windows/win32/winmsg/about-messages-and-message-queues#system-defined-messages
    private val WM_NCCALCSIZE = 0x0083
    private val WM_NCHITTEST = 0x0084
    private val GWLP_WNDPROC = -4
    private val margins = WindowMargins(
        leftBorderWidth = 0,
        topBorderHeight = 0,
        rightBorderWidth = -1,
        bottomBorderHeight = -1
    )
    private val USER32EX =
        runCatching { Native.load("user32", User32Ex::class.java, W32APIOptions.DEFAULT_OPTIONS) }
        .onFailure { println("Could not load user32 library") }
        .getOrNull()
    // The default window procedure to call its methods when the default method behaviour is desired/sufficient
    private var defaultWndProc = if (is64Bit()) {
        USER32EX?.SetWindowLongPtr(windowHandle, GWLP_WNDPROC, this) ?: LONG_PTR(-1)
    } else {
        USER32EX?.SetWindowLong(windowHandle, GWLP_WNDPROC, this) ?: LONG_PTR(-1)
    }

    init {
        enableResizability()
        enableBorderAndShadow()
        // enableTransparency(alpha = 255.toByte())
    }

    override fun callback(hWnd: HWND, uMsg: Int, wParam: WPARAM, lParam: LPARAM): LRESULT {
        return when (uMsg) {
            // Returns 0 to make the window not draw the non-client area (title bar and border)
            // thus effectively making all the window our client area
            WM_NCCALCSIZE -> { LRESULT(0) }
            // The CallWindowProc function is used to pass messages that
            // are not handled by our custom callback function to the default windows procedure
            WM_NCHITTEST -> { USER32EX?.CallWindowProc(defaultWndProc, hWnd, uMsg, wParam, lParam) ?: LRESULT(0) }
            WM_DESTROY -> { USER32EX?.CallWindowProc(defaultWndProc, hWnd, uMsg, wParam, lParam) ?: LRESULT(0) }
            else -> { USER32EX?.CallWindowProc(defaultWndProc, hWnd, uMsg, wParam, lParam) ?: LRESULT(0) }
        }
    }

    /**
     * For this to take effect, also set `resizable` argument of Compose Window to `true`.
     */
    private fun enableResizability() {
        val style = USER32EX?.GetWindowLong(windowHandle, GWL_STYLE) ?: return
        val newStyle = style or WS_CAPTION
        USER32EX.SetWindowLong(windowHandle, GWL_STYLE, newStyle)
    }

    /**
     * To disable window border and shadow, pass (0, 0, 0, 0) as window margins
     * (or, simply, don't call this function).
     */
    private fun enableBorderAndShadow() {
        val dwmApi = "dwmapi"
            .runCatching(NativeLibrary::getInstance)
            .onFailure { println("Could not load dwmapi library") }
            .getOrNull()
        dwmApi
            ?.runCatching { getFunction("DwmExtendFrameIntoClientArea") }
            ?.onFailure { println("Could not enable window native decorations (border/shadow/rounded corners)") }
            ?.getOrNull()
            ?.invoke(arrayOf(windowHandle, margins))
    }

    /**
     * See https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-setlayeredwindowattributes
     * @param alpha 0 for transparent, 255 for opaque
     */
    private fun enableTransparency(alpha: Byte) {
        val defaultStyle = User32.INSTANCE.GetWindowLong(windowHandle, GWL_EXSTYLE)
        val newStyle = defaultStyle or User32.WS_EX_LAYERED
        USER32EX?.SetWindowLong(windowHandle, User32.GWL_EXSTYLE, newStyle)
        USER32EX?.SetLayeredWindowAttributes(windowHandle, 0, alpha, LWA_ALPHA)
    }

    private fun is64Bit(): Boolean {
        val bitMode = System.getProperty("com.ibm.vm.bitmode")
        val model = System.getProperty("sun.arch.data.model", bitMode)
        return model == "64"
    }
}

To minimize the window, instead of using (window as? ComposeWindow)?.isMinimized = true use User32.INSTANCE.CloseWindow(windowHandle):

@Composable
fun WindowScope.MyCustomMinimizeButton() {
    val windowHandle = remember(this.window) {
        val windowPointer = (this.window as? ComposeWindow)
            ?.windowHandle
            ?.let(::Pointer)
            ?: Native.getWindowPointer(this.window)
        HWND(windowPointer)
    }
    Button(onclick = { User32.INSTANCE.CloseWindow(windowHandle) }) {
        Text("Minimize")
    }
}

I also added the following rule to my Proguard rules.pro file for the app release version to work correctly:

-keep class com.sun.jna.** { *; }

[mahozad]

@pjBooms Fixed the problem with native animations and shadows as described in the above comment.

Can they be incorporated into Compose Multiplatform code?

[pjBooms]

@mahozad currently we are busy with other tasks

[mahozad]

OK, no problem.

[sheng-ri]

@mahozad I tried this,it works, but I can't resize the window.

[mahozad]

@sheng-ri My app did not need to be resized (I set resizable = false). Sorry.

[sheng-ri]

@sheng-ri My app did not need to be resized (I set resizable = false). Sorry.

I found a solution for resizable window base your code. You need make window style has WS_CAPTION. .

[sleinexxx]

@sheng-riРазмер моего приложения не нужно было изменять (я установил resizable = false). Извини.

Я нашел решение для изменения размера окна в вашем коде. Вам нужно, чтобы стиль окна имел WS_CAPTION. .

how to set this style?

[sheng-ri]

@sleinexxx

See win32 api
You can convert this to use JNA. Here a example(Using FFM):

final var style = (long)GetWindowLongA.invoke(hWnd, GWL_STYLE);
SetWindowLongA.invoke(hWnd, GWL_STYLE, style |  WS_CAPTION);

[sleinexxx]

@sleinexxx

See win32 api You can convert this to use JNA. Here a example(Using FFI):

final var style = (long)GetWindowLongA.invoke(hWnd, GWL_STYLE);
SetWindowLongA.invoke(hWnd, GWL_STYLE, style |  WS_CAPTION);

I tried to do this, but it didn't work. WS_SIZEBOX too

[sheng-ri]

I tried to do this, but it didn't work. WS_SIZEBOX too

@sleinexxx I forget one thing,This solution need set undecorated=true I still can't undertstand that, but seems this works.

[sleinexxx]

I tried to do this, but it didn't work. WS_SIZEBOX too

@sleinexxx I forget one thing,This solution need set undecorated=true I still can't undertstand that, but seems this works.

it works! thanks a lot

[Sanlorng]

No undecorated = true required

Hit test handled by SkiaLayer's Canvas，See TransparentDecorationWindowProc if hit non client area return HTTRRANSPARENTthat will pass the event and make ComposeWindowhandled the NCHITTESTevent，then we return the non client area hit test in the CustomWindowDecoration. Another issue was HTMAXBUTTONwill intercept the mouse event that will cause custom maximize button can't recieve the click event, so we need redirect the mouse event when hit area is HTMAXBUTTON Skia Layer hit test

    override fun callback(
        hwnd: WinDef.HWND,
        uMsg: Int,
        wParam: WinDef.WPARAM,
        lParam: WinDef.LPARAM
    ): WinDef.LRESULT {

        return when(uMsg) {
            WM_NCHITTEST -> {
                val x = lParam.toInt() and 0xFFFF
                val y = (lParam.toInt() shr 16) and 0xFFFF
                val point = POINT(x, y)
                USER32EX?.ScreenToClient(windowHandle, point)
                val offset = Offset(point.x.toFloat(), point.y.toFloat())
                point.clear()
                hitResult = hitTest(offset)
                when(hitResult) {
                    // 1 is client area, 9 is HTMAXBUTTON
                    1, 9 -> LRESULT(hitResult.toLong())
                    else -> LRESULT(-1)
                }
            }
            WM_NCMOUSEMOVE -> {
                //dispatch the mouse event to custom maximize button
                USER32EX?.SendMessage(contentHandle, WM_MOUSEMOVE, wParam, lParam)
                WinDef.LRESULT(0)
            }

            WM_NCLBUTTONDOWN -> {
                //dispatch the mouse event to custom maximize button
                USER32EX?.SendMessage(contentHandle, WM_LBUTTONDOWN, wParam, lParam)
                LRESULT(0)
            }

            WM_NCLBUTTONUP -> {
                //dispatch the mouse event to custom maximize button
                USER32EX?.SendMessage(contentHandle, WM_LBUTTONUP, wParam, lParam)
                return LRESULT(0)
            }

            else -> {
                USER32EX?.CallWindowProc(defaultWindowProc, hwnd, uMsg, wParam, lParam) ?: WinDef.LRESULT(
                    0
                )
            }
        }
    }

finally: i implement the window decoration like the video shows.

https://github.com/JetBrains/compose-multiplatform/assets/26089739/750ee5a7-7126-41eb-b5b0-1c94fda402c0

[mahozad]

@Sanlorng Can you please share how you obtain the contentHandle? and your hitTest() function?

or even better, share your whole code for window decoration?

[Sanlorng]

@Sanlorng Can you please share how you obtain the contentHandle? and your hitTest() function?

or even better, share your whole code for window decoration?

I will share it when i rearrange the code

[Sanlorng]

@Sanlorng Can you please share how you obtain the contentHandle? and your hitTest() function?

or even better, share your whole code for window decoration?

contentHandle is HWND(SkiaLeyer.canvas)

[Sanlorng]

@Sanlorng Can you please share how you obtain the contentHandle? and your hitTest() function?

or even better, share your whole code for window decoration?

@mahozad i open a Pull Request which include the whole implementation in the one commit and reference this issue, you can fork the code at your app and try it

[mahozad]

It works! Thank you. Update: for the most part :)

Here is your pull request link: https://github.com/Konyaco/compose-fluent-ui/pull/57

[okushnikov]

Please check the following ticket on YouTrack for follow-ups to this issue. GitHub issues will be closed in the coming weeks.