stacked layouts which fill an entire output do not respect `layout default` but they do in i3wm

[kevenwyld]

As stated in the title. When you create a stacked layout that fills an entire output you can not "unstack" the layout using layout default. This is possible in i3wm.

Steps to reproduce:

1. Open an empty workspace on an output
2. Create two windows which will tile in the default manner (probably horizontally if your output is standard orientation).
3. Stack the layout using layout stacking keybinding
   • The two windows will become a stack filling the entire output
4. Attempt to "unstack" this layout using a layout default keybinding.

Expected behavior:

The layout will return to step 2 where there are two horizontally tiled windows

Observed behavior:

Nothing, the windows remain stacked and layout default has no impact.

Additional notes:

1. If you stack windows that do not fill the entire output layout default works as expected.
2. If you add another tiled window adjacent to the container created in step 3 layout default still does not work on the stacked container. This is also different behavior from i3wm.

Using swaymsg to issue the command manually in step 3 reports the following:

[ hostname~ ] > swaymsg "layout default"
Error: Expected 'layout default|tabbed|stacking|splitv|splith' or 'layout toggle [split|all]' or 'layout toggle [split|tabbed|stacking|splitv|splith] [split|tabbed|stacking|splitv|splith]...'

Using swaymsg to issue the command on windows that do not fill the whole output works as expected with no output.

So the help output is saying "Error: Expected 'layout default'" however that's what was passed.

Here is a short (and unfortunately rather bad quality) screen capture of the problem:

https://github.com/swaywm/sway/assets/11936535/62ab8da0-92ff-43fd-8b1a-f390646c21e6

Thanks for all the hard work you all have done on this project!

EDIT: Forgot to include my sway version:

sway version 1.8.1

[kevenwyld]

hmm, maybe this should be a bug instead of the i3-compat label given the output from swaymsg. Feel free to change if that fits better.

[kevenwyld]

I just discovered that if I use layout toggle stacking split instead of layout stacking when I create the stack then layout default works as expected. But if I use layout stacking then layout default does not work producing the error:

Error: Expected 'layout default|tabbed|stacking|splitv|splith' or 'layout toggle [split|all]' or 'layout toggle [split|tabbed|stacking|splitv|splith] [split|tabbed|stacking|splitv|splith]...'

This is an OK workaround since running layout toggle stacking split only one time in a container of splits still produces a stack of the contents, I just have to remember not to press it again. Though honestly doing this with one binding is probably better anyway and I'll likely move to that instead.