Design/implement API for third party "applets" in dock, panel, etc.

[ids1024]

We want to make Cosmic extendable. Supporting the kind of custom scripting Gnome Shell offers is likely undesirable, so the obvious alternative is to support "Applets" that can be added and placed on a dock or panel.

It's worth looking at how KDE, XFCE, Mate, etc. implement this sort of thing. But there are a few ways it could work:

• Implement the applet in a separate process
  • Seems good in theory, but Wayland lacks a protocol like XEmbed to embed one application in another. xdg-foreign allows an xdg_toplevel from one application to set its parent as an xdg_toplevel of another, but there doesn't seem to be such a mechanism for an xdg_popup.
• Statically compile in all applets
  • Probably not an option if we want users who are not developers to be able to install a custom applet
• WebAssembly
  • The correct option if we want any possible security against a malicious extension, but it's not obvious how graphics from that would be embedded either, and it would be limited to any APIs the system exposes
• Dynamic libraries
  • This seems to be the practical solution. Agnostic to the backend (X or Wayland), and I think glib has the necessary support:
  • https://docs.gtk.org/gobject/class.TypeModule.html
  • https://docs.gtk.org/gmodule/modules.html

[Drakulix]

but there doesn't seem to be such a mechanism for an xdg_popup.

Why would this be a problem? Popups are just meant to be short lived surfaces with special positioning requirements. I would not expect any applet to be a popup.

That said, nothing keeps the process from attaching a popup to the toplevel, that is exported onto the dock/panel/whatever. xdg-foreign does not prevent nesting.

I think implementing this in a separate process is the most robust solution out of the four. And it also does not require the use of some c-api for exporting/importing symbols.

[ids1024]

Why would this be a problem? Popups are just meant to be short lived surfaces with special positioning requirements. I would not expect any applet to be a popup.

I was thinking of the possibility of drop downs that are implemented out of process. The applet it self would have to be a simple label and/or icon then.

I'm not sure exactly what you have in mind. The "Embedding Compositor" section of https://wayland.freedesktop.org/docs/html/ch02.html suggests something like a panel could embed its own compositor for this sort of use. Not sure how it would work in practice.

And it also does not require the use of some c-api for exporting/importing symbols.

I don't think it would be too hard to use the dynamic loading support in glib for this. And it would have the benefit of working exactly the same way on X.

But avoiding dynamic loading and preventing applets from blocking the panel's main thread could definitely be beneficial.

[Drakulix]

I'm not sure exactly what you have in mind.

I had to read up on xdg-toplevels and xdg-foreign again and I think I get the problem now. There is no way to express the positioning of foreign surfaces, because the semantics of xdg-toplevel.set_parent apply.

Just looking at the interfaces of xdg-foreign, I though we could just create a subsurface for each client and export that surface via xdg-foreign.

If we control the compositor, we could just allow xdg-foreign to be used with non-toplevel surfaces. Alternatively we could try to push for another xdg-foreign version, that allows this to match the possibilities of XEmbed, or design a custom-protocol in that vain.

But avoiding dynamic loading and preventing applets from blocking the panel's main thread could definitely be beneficial.

This was my main motivation to do this out-of-process. It also limits our api surface, given that this leaves all unnecessary complex tasks like buffer-management and compositing to the compositor.

The "Embedding Compositor" section of https://wayland.freedesktop.org/docs/html/ch02.html suggests something like a panel could embed its own compositor for this sort of use. Not sure how it would work in practice.

That solution sounds a lot like unnecessary complexity, but given the very limited set of protocols this compositor would actually need to support, this could be another viable solution. And this solution would also work on X, while the clients could all be wayland-native.

[mmstick]

It could be achieved with ECS and IPC without the need for dynamic libraries if some standard widgets are provided that can be declared by the plugin, but managed solely by the shell.

let appindicator_id = service.widget_add(ShellWidget::AppIndicator {
    icon: "/path/to/icon",
    text: None,
    on_click: "show-context"
});

Then the shell can signal the plugins about activities it needs to respond to

let button_id = service.widget_add(ShellWidget::Button {
    icon: None,
    label: "Click Me",
    on_click: "do-this"
});

let list_id = service.widget_add(ShellWidget::List {
    items: vec![button_id]
});

let popup_id = service.widget_add(ShellWidget::Popup {
    parent: appindicator_id,
    content: list_id,
});

while let Some(event) = rx.recv() {
    match event {
        ShellResponse::Custom("show-context") => {
            service.show(popup_id);
        }
        ShellResponse::Custom("do-this") => {
            service.show(dialog_id);
        }
    }
}

---

Dynamic libraries may present some security risk, but it is entirely possible in Rust now with abi_stable.

[Drakulix]

It could be achieved with ECS and IPC without the need for dynamic libraries if some standard widgets are provided that can be declared by the plugin, but managed solely by the shell.

Also a very viable solution, but I still feel like drawing should be the responsibility of the client not of the panel. Otherwise we might end up with either a rather large widget library or some kind of bitmap-widget the client can draw into whatever it wants. And if we give the clients the possibility to render anything, why not just make them full-blown wayland clients.

But avoiding dynamic loading and preventing applets from blocking the panel's main thread could definitely be beneficial.

So to solve this we would need (or want) to spawn a dedicated thread per client? I expect most panel elements will want to poll some data to display and thus need to block for that anyway. Which then leads to the question, if those threads might need an event loop if they get more complex or to integrate with the panel. Or maybe the panel should provide this event loop?

I think there is lot to be gained here by externalizing clients into separate processes and not having to deal with all of this. If we do this via some variant of xdg-foreign on wayland or xembed on X to just re-parent their surfaces onto a dedicated space inside our panel or dock, we basically just need an api to request a region inside the panel/dock and a response that contains a window-handle. This could be for example a very simple dbus protocol.

[ids1024]

If we control the compositor, we could just allow xdg-foreign to be used with non-toplevel surfaces. Alternatively we could try to push for another xdg-foreign version, that allows this to match the possibilities of XEmbed, or design a custom-protocol in that vain.

I think we probably want basic functionality like this to be relatively compositor agnostic. So a custom protocol no one else would support is possible but probably not what we want.

if some standard widgets are provided that can be declared by the plugin, but managed solely by the shell

Not sure how many widgets would be necessary for most use cases. It would be hard to provide enough functionality for anything one might want. For instance, how many widgets/events/etc would you need to implement the popover Gnome implements when clicking the clock, with a calendar, notifications, media controls, weather.

It also would amount to a somewhat different and more limited toolkit to learn. I'm hoping one of the advantages of developing things like applets for Cosmic over Gnome Shell would be that you could just use any gtk4 widgets, instead of being stuck with something like Clutter/St.

[Drakulix]

I think we probably want basic functionality like this to be relatively compositor agnostic. So a custom protocol no one else would support is possible but probably not what we want.

In that case we might want to reach out and figure out if there is interest in extending the scope of xdg-foreign.

EDIT: https://gitlab.freedesktop.org/wayland/wayland-protocols/-/issues/74

[ids1024]

I guess we're now planning to use an embedded compositor, assuming all goes well in testing.

With this design, an applet is a Wayland application that creates a single xdg_toplevel, without decoration, and is given a fixed height and/or width to fit within.

• Perhaps orientation of the panel can be passed as an env var so the applet can know about it.
• We expect to write applets using gtk4-rs, but with this design, one could be implemented with another toolkit that supports Wayland, or even direct Wayland calls without a toolkit.
  • Useful for users wanting to add an applet with their preferred language, and provides flexibility if some things are easier to implement without GTK4. Or it allows incremental migration to another toolkit.
• libcosmic probably should provide some convenience wrapper for developing applets.
  • CosmicApplet subclass of GtkWidget, with properties for width/height, panel orientation. Automatically calls set_decorated to disable window decoration. Maybe have a run method to start main loop.
• How to handle panel background color? GTK can use transparent background just fine, but if background color can change, need to make sure colors used in applet match.
• By running each applet in a separate process, things like applet crashes can be detected, to log the issue or restart the applet.