Neos-Metaverse / NeosPublic

A public issue/wiki only repository for the NeosVR project
193 stars 9 forks source link

Automatic context menu injection #1576

Closed Earthmark closed 3 years ago

Earthmark commented 3 years ago

Is your feature request related to a problem? Please describe.

Users require menus for custom toggleable behaviors, such as emotes and avatar systems. These are currently done through custom private UI's, however this leads to a wide diversity of systems that are difficult for new users to learn, and leads towards a classic linux like culture where people are building their own shells.

This isn't a problem for dev-oriented users, however non-dev-oriented users may get lost, or are stuck without support for features they often consider common in other platforms due to a lack of standardization.

Relevant issues

None found.

Describe the solution you'd like

For a more requirements based look rather than this brick of text, see https://github.com/Frooxius/NeosPublic/issues/1576#issuecomment-758228454

I would like a component added, that when enabled inserts an Arc context menu entry into the users context menu when opened that routes click events to an external location where the developer can do great and terrible things with that event. That context menu item should behave like any other context menu item, including drag selection (which does not work with some of the current context-menu hacks).

The created menu item is the standard format of context menu item, and is indistinguishable besides a TouchEventRelay that routes to the ContextMenuRegistration (if that's how they work, I think that's how they work where they redirect an event to another component).

A part of this is also that context menu items must be injectable from elsewhere in a world, for instance a world may have a management subsystem that registers a context menu submenu into the host's context menu.

Behaviors are enabled with IButton on the registration, the context menu arc is a standalone entity that is generated by the system itself. This way runtime duplication is to a minimum and the arc and update over time should the UI go through more revamps.

I see the component working with the following fields: ContextMenuRegistration : IButton I get a strong feeling I'll get pushback on this interface choice cause it's not a button, it represents a possible button, but it works usage wise and allows UI events that enable a lot of flexibility with the existing interaction components.

Submenu hierarchies

Submenus are a bit more of an odd feature, however without them if a user wanted to handle submenus they would need events that change enabled state to run before a click event occurred, and then be able to re-open the context menu of a user which sounds very complex and error prone (if not outright impossible due to execution order). This is also one of the primary goals of this proposal, as it would make submenus more accessible for categorized use, hopefully simplifying some of the more context-arc spammy things.

Submenu Tag Approach One possible approach is to have submenus be represented by fields on the ContextMenuRegistration itself, here is an attempt to support context menu nesting using a pattern based on one I saw Hayden using. I found it quite flexible and easy to work with personally, so it may be a good choice. If an enabled ContextMenuRegistration exists with a matching Submenu tag, the context menu is re-opened with those found entries available. OF NOTE: Default and Locomotion are available as targets, Default routes back to the root menu. - This can be recursive, this is more of a state machine state change rather than a stack based menu. This facilitates cross menu jumps when required, and also facilitates a back button if required through on click writes to the TargetSubmenu field when a parent menu is clicked.

ContextMenuGroupApproach Another approach is ContextMenuGroups, this would be a component that can be invoked via an action like clicking a button, or an impulse. On fire it re-opens the users context menu with the specified group. I haven't thought over this approach too much, it was brought up by @mralext20 as a possible approach and would require different sets of fields for the ContextMenuRegistration, as the group would take over for the HideForTool and Hand categories, the context group would be the conditions for the registration being shown, instead of the registration having some of that information.

I can work out those fields as well, but this was a viable alternative to allow submenus, which feel like a critical part of this.

Describe alternatives you've considered

Context menu injection systems exist, however getting them to work is often non-trivial and normally error prone, such as the lack of drag selection support. Some context menu systems require additional info to be inserted into the menu as well, such as logix to forward events.

Making the context menu easily injectable will make it a go to for new users trying to add behaviors, instead of the invisible cube frenzy that tends to happen with a bunch of private UI TouchToggle cubes.

Custom UI approaches are also available, however that it is a new experience and UI for each tool makes it problematic for new users who expect a common interface. This may be an incorrect approach for some new users, however it is a common one.

Additional context

My hope with this is to make context menu injection safer and more future proof, if registrations are done through a single component, the UI can flex towards whatever it needs to over time, should theming or such effect the context menu. This also helps make it easier to address some desktop usability concerns with some custom menus, as they can more easily provide context menu support for desktop mode for behaviors that may be unusable when parented directly to an avatar.

This approach could also allow pagination in context menus, if 20 registrations are made to the menu, the menu handler would be able to page them into the menu automatically, as well as adding the forward and back buttons. This would resolve usability issues that arise from some tools injecting a few dozen entries into the context menu, making the buttons impossibly small and unreadable.

Earthmark commented 3 years ago

I'm going to look further into the menu group approach, that has a lot of promise around clicking on things to open a context menu. So more objects could have behaviors like the equip avatar button does. As an example, a light switch could open a context menu with more advanced options, instead of needing to make an ingame menu for it.

Earthmark commented 3 years ago

Submenus - ContextMenuGroup Approach

After pondering more, here's a second approach that should provide more utility than the tag approach.

In this approach a second component would be added as well, ContextMenuGroup.

Each group has a list of ContextMenuRegistration references, and when actived the invoking user's current context menu is closed if opened, and a new context menu with the items specified by the context menu group is opened.

A context menu group can be activated through a number of possible means, the most probable is an interaction component and/or a logix node. The logix node would be more configureable, as it would allow impulses to open menus, however the interaction component (like a button set event) would allow logix-less context menus.

In this system submenus would be created by having a ContextMenuRegistration on a menu with the interaction component on the same slot. Then when the arc created to represent the registration is pressed, the touch event is proxied to the registration, into the OpenContextMenu component, replacing the current open context menu.

The logix approach would work as well, but the component form supports submenus without the need for logix at all.

ContextMenuGroup

Special cases and how this changes the ContextMenuRegistration

The default submenu is a special form of a ContextMenuGroup, a ContextMenuGroup has to be manually activated, however another component called DefaultContextMenuGroup can be added that will add referenced context menu items to the menu when the menu button is hit on root. This would replace the Enabled, User, HideFortool, Hand, and OrderHint fields on ContextMenuRegistration, as they would be owned by this component instead.

Enabled doesn't make sense as the registration isn't actually registering anything anymore, it's representing a static possible entry only.

User doesn't make sense for the same reason Enabled doesn't, nothing is registering anymore and actions would be invoking on a specific user based on component events or logix impulses (which are user bound).

Hide for tool is only relevant for DefaultContextMenuGroup, as other submenus are explicitly constructed (at least right now?).

Hand is only really relevant for DefaultContextMenuGroup as other interactions would invoke (and be assumed to invoke) related to the hand that caused the invocation target (or face for desktop users). This is hopefully possible as clicking on an avatar is already able to tell the correct hand to show the context menu on, and this would hopefully reuse that same behavior. Some interaction like nose touch tip sources can get complicated here, however in those cases it should likely use the default user hand.

OrderHint is only revelant for the DefaultContextMenuGroup because ContextMenuGroup itself would use the order of items provided in MenuItems.

The new proposed, and modified ContextMenuRegistration is as follow, it is also renamed to better represent what it as as in this modified proposal only DefaultContextMenuGroup is registering things. ContextMenuProxy : IButton

The proposed DefaultContextMenuGroup takes on the missing fields, making it DefaultContextMenuGroup

Other notes - Disclaimer of not being pushy

I forgot to mention this before, but of course this is just a concept and I've got no idea how the engine internals work and such. I'm outlining what I see as requirements and what I'd hope to see as an API based on what I'd like to do. If other things are required or this is totally against the goals of the menu, that's fine by me! I'm just addressing a concern.

Other notes - Submenu Tags

Tags may still be useful in this context, so ContextMenuGroups can auto group together, however I'm not sure if that's a good idea and it would complicate the pattern. The pros of adding tags would be submenus could auto-aggregate, so the OpenSubmenuAction whatevers could pass in a string, and the system assembles the submenu entries.

To do this, in my mind, would mean that the fields on DefaultContextMenuGroup would be added to ContextMenuGroup, and the MenuTag: String field would be added. DefaultContextMenuGroup would not exist, and instead be a ContextMenuGroup with a MenuTag of "Default". However, there's a good chance this is now adding the group component to just bundle a bunch of entries together that would otherwise have the same fields, which is not great.

Earthmark commented 3 years ago

I realize I probably got too specific tbh, but I hope it better explains what my perceived requirements are! Let me know if these requests should be more generic instead of requesting specific fields.

TehTurk commented 3 years ago

I really really like the idea of this, it also streamlines it and doesn't have everyone's different setups conflict but the one question I surmise is how to access it in terms of usability from a UI Interaction perspective🤔

Currently the only other official context menu action would the desktop mirroring where you need to click on the panel to access stuff like keyboard, other monitor etc. And say if you hit a light switch or need some strong identifier to know when something is going to have a different MenuGroup like say in a world in some capacity. The Context Menu in of itself is a bit foreign to most when just joining.

Tho Ideally, I think having the Context Menu appear at the point of interaction, or a predefined space with filling out the ArcLayout under the Radial Menu with Said Custom Buttons/Menus would be Ideal.

I also want to bring up the hand facets as we still in-between the UI-Update to some degree.

I'm very much for the MenuGroup Approach as that's very much how I coded my context Menu System #soon, so instead it reads as a list of whatever extra tools/access you might want.

Earthmark commented 3 years ago

That was the wrong button...

TehTurk commented 3 years ago

That's ok Mobile Github is a landmine xD

Earthmark commented 3 years ago

With regard to the desktop viewer, that was not trying to be poked by this change,v I'll try to summarize:

There are two added paths I'm requesting, and I'm trying to request them in a way that doesn't require updating and is less error prone than the current approach

1 - Provide the ability to open a context menu with only custom entries

This is for anything that requires a custom context menu behavior, for instance a menu button on a wall, or a different context menu button (see 2).

These buttons must be system generated to allow better standardization, and the button events must be routed to a space the developer can easily consume them.

2 - Provide the ability to add items to the default context menu

This is for things that may not have a physical world representation, such as avatar config or global world settings.

Actions added in this way may invoke 1.

Electronus commented 3 years ago

When official context menu injection is added, being able to use the injected arc in spectator would be nice, for things like avatar control.

For example the wings on my avatar are largely automatic, but occasionally are in a undesired state, or the logic breaks. When that happens I have buttons on my arm to override their state, being able to do this through the context menu even while spectator would be a very nice feature.

Frooxius commented 3 years ago

Added in 2021.5.25.1225! The system is relatively simple on itself, but leverages existing IButton events, so it should be pretty flexible and powerful!

Added Context Menu Injection system for adding custom items (components under Radiant UI / Context Menu) (requested by @Earthmark, @Turk | Mentor Manager, @Electronus and GitHub / Patreon Priority Voting)