How to move yabai into the future? ideas / visions / cross OS / nested layouts / new JS like action/selection language

[MarcWeber]

First of all I know that I could be scripting it all up using signals etc. But maybe it's not the best design pattern.

Imagine yabai would have layout plugins like this:

layout_plugin = {
  name
  activate_for_space : () => initial layout of windows,
  deactivate_for_space : () => initial layout of windows,
  new_window : () => .. // new window appeared, adopt layout
  drag_drop: { hover: (coord) => ., dropped: () => ..}
  selectors: ..
  command: (command: string[]) => ...
}
and some state tracking like recent window(s), so that an appearing window can
be put into column of last active window

Then a wmii like or other plugin could be written to place windows the way you want.

Mostly I use EDITOR | BROWSER like here:

So there are columns, you can add windows to. Then you have different ways to display: evenly spaces (can be changed), stack like (one is visible) or stacked with labels.

I ended up using only the stacked maximized version recalling from memory always having 'terminal' below my editor or full screen mode if I need more.

So main question is does it make sense to allow writing layout managers as plugins eventually adding a scripting language ?

So a plugin basically would be a selection of signals and behaviors chosen by the user to be active for a space which once chosen to be active rearrange the windows according to the plugins.

WMII ships with wmii menu. It allows to tab complete existing words and enter own text. Perfect for creating 'spaces with label' or reuse if it exists. What might be the easiest way to get such feature ?

[koekeishiya]

"Layouting" is very much an opinionated thing, and there is inherently nothing wrong with a plugin system like you are suggesting. I imagine it would be quite a bit of work to implement this robustly on top of the current yabai model though.

With that said.. The layouting performed in yabai is tightly coupled to the bsp system, which I think is a mistake today. It was built based on my experience using bspwm on Linux in the past, and I also wouldn't be using bspwm today.

The best way I can describe the problem is that it is highly flexible, in a bad way. It works very well for daily ad-hoc usages where you use 1 or 2 (maybe even 3) windows per space and open/close windows regularly (browser, editor, terminal, video player). Anything more sophisticated and it goes out the window.

I have various ideas I'd love to try do make things work very differently, but can't really implement on macOS because we are not fully in control of the window server.

I have not been very interested in this kind of work for a long time, so this project is mostly in a maintenance state and has been so for a long time at this point.

[MarcWeber]

I'd be interested in your visions :-) (maybe VR and 3d :-)

AFAIK:

• you have notifications (move, resize, new, destroy) of windows
• you can move/resize windows -> this is enough to implement everything you dream of

But to be honest I'd love to see this for Windows/Linux, too :-) So a cross platform API would be fun to have.

I came up with the following sketch thinking it through. I agree it would be a bigger change cause it would not be backward compatible. But eventually backward compatibility could be maintained.

Allow nested layouts. Change the rigid move warp whatever into JS like language. Basically the object is either window or layout. Then you can walk the tree by methods. (like jquery) Implement layout specific features like rotate90 by sending the command to it as string. (should be flexible enough) Same for moving. Eg moving beyond first means escalating to parent Layout ?

Then you can tabs, stacks, WMII like columns etc implemented maybe easily. Eg special behavior moving right(east) within WMIIColumns means adding new column and placing window into it.

So even nested Stuff Like WMiiColumns ( _, WMIIColumn( Tabs ( win ))) moving win to the right escalates Tabs and WMIIColumn but WMIIColumns will catch and add Column to the right and place window there.

But there is a lot more. Eg if you make windows float in WMII it recalls which column a window came from. So when turning a window into float mode it could reference Layout parents to which it could be attached again when unfloating

example:

Each container contains windows or layouts. So a layout is a container.

interface layout: {
  create: (parent, win_ids or old_layout) -> when switching to this layout

  add_window:

  containter_and_windows : list of containters and or windows for iteration

  change_layout : (win_Id or container_id, new_layout)

  // user used mouse to move ! try to cope and adopt
  window_moved (win, old_pos, new_pos) => .. resize everything or bubble up

  window_move_take_over_from_inner: called from inner layout if a window gets moved north leaving the inner container

  // unique interface for selecting or moving windows or other state changes
  // user's keyboard actions
  // will bubble from the inside out or such till some layout handles it ?
  // of course some CSS like selector or such would be imaginable such as
  // LayoutWMIIColumns>window.move.north or LayoutStack>window.move.north
  // or focused.-3... would mean go 3 levels up from focused ..
  action: 
    select.(next|prev|north|.3)
    move.(next|prev|north|.3)
    rotate90
    containertype.XX switch to containertype XX   -> #dis

}

implementations

LayoutDisplays { // for each connected display hold one container
  container_for_each_display: ..
}

LayoutWMIIColumns {
}

LayoutWMIIColumn {
}

LayoutBsp {
}

LayoutTabs {
}

LayoutStack {
}

LayoutNone {
  // This basically is floating user needs to positino windows, don't touch
  // their location at all
}

LayoutFixedDisplay

So WMII would be

LayoutDisplays {
    floating: [win10, win11]

    // Monitor1
    LayoutFixedDisplay{
      LayoutWMIIColumns {  // when moving left/right ouf outer columns, adds new column
        LayoutWMIIColumn { // when moving window right/left hands over to parent
          win1, win2, win3
        }
        LayoutStack { 
          // moving top/bottom changes order, when moving beyond first/last
          // hands over to parent which fails in this case 
          // because WMIIColumns only handles moving left/right
          win20, win30
        }
      }
    },

    // Monitor2
    LayoutFixedDisplay{
      // no window on snd monitor, so no layout chosen yet ?
    }

  ]
}

#dis if you allow nested containers having one window focused within containers

Imagine you have this layout

Layout2Columns {
  LayoutBsp
    win1
    LayoutTabs
      win2, win3(SELECTED)

  LayoutFloat (within right half of 2columns you can position windows you as you want)
    win5, win10
}

Now you click win3, and want to change LayoutBsp to LayoutWMIIColumn.
Then yabai could imagine walking up the tree and selecting the layoutmanagers
till you have selected the one you want to change. So the state of yabai would be
  yabai_selected: window or invisible layout manager
  osx_selected: the window

Introducing CSS like or JS like navigation would make everything simple:

  osx_selected.select_parent().change_layout(LayoutWMIIColumn)
  osx_selected.walk_till_instance(LayoutWMII).change_layout(...)
  osx_selected.send('move:north')
  osx_selected.opacitiy(0.2)
  osx_selected.change_layout(LayoutWMIIColumns) -> if window, change into layout, then add window

...

[MarcWeber]

https://github.com/koekeishiya/yabai/issues/193 might be another use case for adding additional layers to manage/swap spaces on multiple displays at the same time. Cause I want to focus on a topic which might occupy 2 displays. What I noticed is that warping window west doesn't warp on second display. So you cannot move a window using warp west to the free display. With WMII this is exactly what happens. You keep moving to next column, if the screen is free it will just be added one and the window will be moved there.

[MarcWeber]

WMII behavse the following way: maximize window -> make it float full screen The advantage is that if you need more windows eg when browsing you can cycle all floating windows easily.

And the default bsp binary divisions layout doens't work for me at all. Cause I constantly need to review 200 grep lines, open a browser So I almost always need full height of the screen.

The main question is how easy is it to change the code to allow mulitple layout engines and how to start this process to make everyone happy and other features are on the wish list ?

I don't understand what 'patching dock' acutally works and how the pieces work together yet. Also do you think adding scripting language makes sense and if so which one? Lua is used often in similar context. But there are also various JS libaries which could be considered.

If you use 2 displays what's limiting moving focus to the other space by keyboard using east/west like commands for instance ?

[MarcWeber]

Here are 2 references to illustrate that Window managegrs on Linux and Windows can be done, too. The question basically is whether we can / want to come up with a design which might serve multiple OS.

https://github.com/qtile/qtile/tree/ff90e5cfae7418ad62d0accc3cb38757fca2cdb9/libqtile/layout columns -> wmii like screensplit / slice -> nested layouts ...

Windows manager: https://github.com/LGUG2Z/komorebi Windows /Rust

[koekeishiya]

Window managers on macOS work fairly differently to window managers on Windows and (especially) Linux. There is just no reason to take on that additional complexity for this project. Linux already has a rich ecosystem for both X11 and Wayland at this point. Windows has a fair share of choices these days as well.

[matdave]

One of my favorite window managers is Forge. I recommend trialing it on a Gnome VM if you have the chance https://github.com/forge-ext/forge

One of my favorite features is the ability to group windows into a set of tabs. It's really helpful when I'm working within a same application on two separate projects concurrently.

[koekeishiya]

What I'd do if I had full control is to create an "infinite grid" where each window would correspond to one tile in this grid. There would be keyboard shortcuts to manipulate the viewport of this grid (move viewport around, increase/decrease the number of visible tiles in any spatial direction), effectively pulling those windows into actual screen view.

Each tile would correspond to a fullscreen window, but as you increase tiles the actual render size of the windows change to accomodate the additional windows that need to be rendered in any direction. If a window escapes screen boundaries due to min size requirements it would be rendered partially, and you could either scroll the grid into view, or have it adjust/shift the viewport automatically upon focus.

[MarcWeber]

Feels like VR/3d :-) But zooming in/out and hardware resources are not well defined eventually. But GPU rendering actually comes close because it only renders the pixels viewed. And VR even has logic to render less pixels in the outer areas of the view because the eye is less sharp there. But if you imagine 20 inkscape istances with SVG graphics and millions of lines you eventually don't want your CPU to waste energy on repainting whenever you zoom :-(. So there is a reason things are still 2d mostly. Maybe requiring less resources. The best work is the one you never do. However all the composing window managers they render using hardware then use compositor logic on graphic card to render the final view (compiz and the many others). For Linux there is arcan allowing to create your own WM . That might be a nice starting point for experimenting with such design. Now problem is 4K on each eye cable to VR -> quality loss. Wifi -> same. And some linux drivers cannot cope well with VR cause they are slower (or my old nvidia K6000 is the problem) but having a lot of lag is bad. There might be a reason Meta focused on Windows. OSX had a fun bubble zoom like thing at the bottom. And many "ZOOM" in features might come close. like having the grid and you just click and zoom. and when zoomed moving the mouse moves everything, too. (anydesk gives that feeling if the other screen is bigger, too). I tried treating my brain as CPU making WMII be the task scheduler: "Next please". So if a compilation takes some time .. next please. block this for 20min (cause waiting for email) .. etc. So having the burden to look at a grid all the time kinda drains me. Same about finding >>word<< icons on windows which changed shape color locations every year. How does Pico4 etc work? The apps mode (eg browser) is limited in size. The "virtual desktop" is total crap because the windows have less than HD resolution. What I had in mind was using the head orientation and nose direction to control focus/ mouse (could be controlling the grid viewport). There is some open source AI software using webcam to learn head direction from video... But there is only so much time / day. If you start imagining 3d you can also place windows like photo slides and whatnot. There is alt-tab something. Like windows it creates screenshots of windows and allows to cycle. So with such images you eventually could get the grid and once you select something you could focus the window or zoom to it. No idea whether it updates the images in realtime. https://arcan-fe.com/2018/03/29/safespaces-an-open-source-vr-desktop/ BTW No need for VR glasses. It can render as window, too. So do you need a grid or sticky windows where new windows stick to nearby windows ? (oh we don't know what tasks belong to each other if you run a web testing .. task where every 5 secs you get a window have fun. On Linux I use Xnest -> problem solved. OSX ??? No idea yet. Maybe quartz ..)

Often you have a task or an idea (a box). Then you open a window (eg browser), then you start hacking (editor). then you want to view the result (app window).. then you need to ... so this feels more like a tree to me than a grid.

I played around with luajit. And I know that you can put a pointer to a struct on the stack and define a C struct definition then access the fields from lua. Luajit is small and fast and available on all important platforms. It can also be used from Python :-) There is a C# port (moonphase) for lua. So it feels like Windows, OSX, and Python (eg qtilte) to get started could be supported.

I tried getting started reading all the code. I am slow. but loop.c eg has concurrency stuff built in.

Can I just replace all the code in the event handlers with calling into lua and start building the window manager there?

There would be one eventual drawback passing pointers from C like into lua: If you access data beyond .. you might get a crash.

I use tabs mostly with shells. And the terminals I use have tabs built in. And tabs in editor. So having tabs isn't most important to me. I end up with 2 columns maybe having 8 windows and if there are more I have an attic column to move windows there.. Sometimes stacking some in float mode cycling them. That eventually is not perfect. But the cool thing is they location of the windows never changes. So I can access them by muscle memory without thinking from memory. Yabay somehow always rearranges the windows which I am not used to (maybe don't want to get used to?).

[chrishoage]

What I'd do if I had full control is to create an "infinite grid" where each window would correspond to one tile in this grid. There would be keyboard shortcuts to manipulate the viewport of this grid (move viewport around, increase/decrease the number of visible tiles in any spatial direction), effectively pulling those windows into actual screen view.

Each tile would correspond to a fullscreen window, but as you increase tiles the actual render size of the windows change to accomodate the additional windows that need to be rendered in any direction. If a window escapes screen boundaries due to min size requirements it would be rendered partially, and you could either scroll the grid into view, or have it adjust/shift the viewport automatically upon focus.

@koekeishiya I haven't had the chance to try it out yet, but I found the Wayland composter niria while ago which sounds like exactly what you're describing

(also 👋 I was a heavy user of chunkwm and kwm back in the day but then switched to Linux ~5 years ago. Now I'm coming back to yabai because my employer said no more Linux)