Application-dependent, context-aware menus

developer91234 commented 3 years ago

(not sure how possible this is under wayland)

The Motivation

The current menus is a cool addition to the desktop UI. However the desktop UI is already probably the most polished and efficient part of the user's interaction. The application UIs are in much worse shape, in addition to being completely inconsistent. Thus it might be worthwhile to have a flie pie menu that changes depending on which application window is raised. This would allow fly pie to essentially extend the UI of individual applications. As I said, the app UIs are the thing that is in desperate need of optimization.

The Solution

Enable the user to:

Create an empty root menu "context_aware_menu"
Draw menu structure "menu_x" if condition x=TRUE.
Draw menu structure "menu_y" if condition y=TRUE.
etc. Thus you create custom menus for different scenatios.

The most useful condition here being "which application is currently in focus?" But I can see a use case for other conditions, like time of day and so on.

Dbusmenuproxy may be useful here to invoke appmenu-items from fly-pie, or even have fly pie draw the menu while hiding it within the application.

The Alternatives

The are no GUI alternatives.

Schneegans commented 3 years ago

Thank you for this suggestion! Conditionally opening menus is definitely something I had in mind since the beginning of the development. I think the approach would be to allow the definition of a set of rules for each top-level menu which need to be fulfilled when the menu is opened. I'll schedule this for Fly-Pie 10 (Fly-Pie 8 is soon-to-be released and there are already enough features planned for Fly-Pie 9 :smile:).

Automatically replacing the application menu (or the context menu, if that is possible) would be awesome in theory, but very difficult in practice. This is mainly due to three reasons:

Existing application menus consist usually of simple text items. Fly-Pie's menus however require an icon for each entry to be used efficiently. Just using the first few letters or something similar for the icon could be possible but would look rather ugly, I guess. Searching a specific, rarely used item in such a menu can take minutes.
Most application menus contain too many items.
Oftentimes, there are entries (especially in context menus) which would ideally be placed in a specific direction. For instance, the go-to-previous-page item in Firefox could be on the left hand side while the go-to-next-page should be on the right. The refresh-page item could be on the bottom of the menu to mimic the behavior of mobile phones where the webpage can be pulled down to refresh. When generated automatically, these items would end up anywhere.

When thinking about this when designing Fly-Pie, I came to the conclusion that the only viable solution is to approach this from the other side. Therefore, Fly-Pie includes a comprehensive D-Bus API which lets you programmatically open menus. This means, you could write a plugin for Firefox which opens Fly-Pie menus! I am not aware of anyone having tried something like this, but it should be possible!

developer91234 commented 3 years ago

Yeah, I guess a the fly pie interface is not a good fit for a full menubar replacement. Picking out specific actions from the menubar and placing them into the fly pie menu could still be an invaluable feature however.

For a full menubar replacement, one could have popup menubar widget with a search box at the top.

-------------
| [search]  |
| ----------- |
| [File]      |
| [Edit]      |
| [Help]     |
-------------

ofc this is outside the scope of your existing UI model but it would be very nice thing nonetheless. It would liberate the global menu idea from the Apple UI model and turn it into just another popup/popup-menu.

Schneegans commented 3 years ago

Yeah, that's definitely not in the scope of Fly-Pie. Do you no Fildem? This is quite similar to what you're looking for, I guess.

Schneegans commented 3 years ago

Here is a screenshot of the HUD feature: https://www.linuxuprising.com/2021/02/how-to-install-fildem-global-menu-and.html

developer91234 commented 3 years ago

Fildem isn't a popup menu, it's a global menu bar. Global menu isn't a good fit for Linux desktop IMO.

Yeah, that's definitely not in the scope of Fly-Pie

It would be within the scope of "popup menu", just not "marking popup menu." So the question is whether you want to include other types of menus - like vertical context menus - under the scope of your project. I think it's an idea worth considering because other types of menus can be superior to marking menus for certain tasks. For example, with context menus, you could open a vertical clipboard history menu at the cursor position.

Schneegans commented 3 years ago

Fildem also contains a HUD which is very similar to what you propose:

In the foreseeable future, I do not plan to integrate other types of menus.

Schneegans commented 2 years ago

Rescheduled as Fly-Pie 12 will be a bug-fix-only release.

Schneegans commented 2 years ago

Rescheduled to Fly-Pie 14, as Fly-Pie 13 will be released soon because it includes an important bug fix.

Schneegans commented 2 years ago

Rescheduled again as Fly-Pie 14 will focus on GNOME 42 support...

luka-n commented 2 years ago

I feel like like this could be a real killer feature, I certainly would love to see it

Schneegans commented 2 years ago

I am currently investigating ways to create a standalone (not GNOME-only) version of Fly-Pie. I have some ideas in mind which could improve the user experience. Context-aware menus will definitely be a core feature of this new version.

blipk commented 1 year ago

Hi @Schneegans would that work on Wayland as well or X11 only?

I'd be interested in adding a simple version of this feature request that simply opens a different menu depending on the currently active window.

I have some experience with working with windows workspaces on my extension https://github.com/blipk/Customised-Workspaces

It'd be great if you could point me to where might be the best place in your code to hook into the keyboard shortcuts (or other menu opening actions) and menu configuration, and any style/contribution guidelines.

I propose adding an application config item for each menu which points to either an xdg application or a window wmclass, and any menus that have the same keyboard shortcut would be preferred by the currently focused window application/wmclass

If you're interested, let me know and I will look into creating a PR.

blipk commented 1 year ago

Actually it looks like the simplest solution was to use this other extension and some DBus calls: https://github.com/hseliger/window-calls-extended https://github.com/ickyicky/window-calls

And then bind a gnome global keyboard shortcut to a shell script with the following:

FOCUSED_WMCLASS=$(gdbus call --session --dest org.gnome.Shell --object-path /org/gnome/Shell/Extensions/WindowsExt --method org.gnome.Shell.Extensions.WindowsExt.FocusClass)
gdbus call --session --dest org.gnome.Shell --object-path /org/gnome/shell/extensions/flypie --method org.gnome.Shell.Extensions.flypie.ShowMenu "${FOCUSED_WMCLASS//[,()\']/""}"

I just then have to name my menus in flypie after the the applications main window WMCLASS, and now I get a different menu for each application with a single keyboard shortcut.

EDIT: Here's an updated version of the launcher script if anyone wants this feature:

#!/usr/bin/env bash
set -e # Stop script execution on any failure
SOURCE_DIR=$( dirname $(readlink -f "$0" 2> /dev/null) )
LOG_FILE="$SOURCE_DIR/menulauncher.log"
FOCUSED_WMCLASS=$(gdbus call --session --dest org.gnome.Shell --object-path /org/gnome/Shell/Extensions/WindowsExt --method org.gnome.Shell.Extensions.WindowsExt.FocusClass)
STRIPPED_WMCLASS="${FOCUSED_WMCLASS//[,()\']/""}"
RESULT_CODE=$(gdbus call --session --dest org.gnome.Shell --object-path /org/gnome/shell/extensions/flypie --method org.gnome.Shell.Extensions.flypie.ShowMenu "$STRIPPED_WMCLASS")
RESULT_TEXT=""
if [[ "x$RESULT_CODE" == "x(-5,)" ]]; then
    RESULT_TEXT="No menu found for application" 
    # Optionally open another default menu here (using above gdbus call)
elif [[ "x$RESULT_CODE" == "x(1,)" ]]; then
    RESULT_TEXT="Menu opened for application"
else
    RESULT_TEXT="Unexpected result when trying to call flypie from dbus"
fi
echo "$RESULT_CODE  $STRIPPED_WMCLASS   $RESULT_TEXT" >> $LOG_FILE

It will log the wmclass for apps to menulauncher.log in the script directory so you know what to name your menus, and optionally you can set a default menu when theres not one for the focused application.

Schneegans commented 1 year ago

@blipk This is an extremely clever use-case of Fly-Pie's DBus interface!! And it's actually pretty easy to set up. Would you still be interested in looking into the code or is this solution sufficient for now? Implementing this into Fly-Pie will be waaay more complicated, I guess :laughing:

blipk commented 1 year ago

Hey @Schneegans thanks, I'd probably just stick with the script for now as it's working well, I'm looking forward to using this with my wacom tablet when that arrives.

It shouldn't be too hard to implement, most of the calls to find the focused window are easy enough, but I'm not familiar with your code so would take me a bit of time to fit it in with your config/setup.

I'm more interested in the version you're working on that's not gnome-dependant.

Schneegans commented 1 year ago

@blipk I published some news regarding the platform-agnostic pie menu! At the end of the post, I wrote down some ways how you could contribute to the project. I would be more than happy if you would follow the progress and support the development along the way :smile:

Schneegans commented 1 year ago

I will not add this feature to Fly-Pie as I am currently focusing on the successor project Kando. Application-dependent menus will become a core part of Kando!

Schneegans / Fly-Pie