Application handling

[lboklin]

Looking for feedback on a concept I'm working on for handling applications represented in the form of cards. None of this is set in stone, including the concept of "cards" as visual representation.

Goal and Purpose

Pave the way for the implementation of features such as keeping track of applications, visualizing them and interacting with them in different ways.

Currently we only handle "anonymous" surfaces handed to us by the Wayland server and we can move them around and rescale them. We need to consider application data for more complex features.

Future features

• Browsing and launching applications (#49)
• Persistent sessions
• Minimizing windows
• Make windows visually fit in with their environment (model with material shader, etc)

UX

A card would basically be visualized .desktop files with icon, name, description, etc. An app launcher would read the user's available .desktop files and allow browsing them like looking through a deck of cards.

App states

• Unopened - visualized in default card form with the icon (or a banner) as "art".
• Open - It's positioned somewhere in the world with its shape expanded and the front covered by the window content (shaded/unshaded weston surface texture).
• Minimized - like default card form but icon gets replaced by a live thumbnail-preview. Can be seen in your "card hand" which slides out from the Vive controller with a controller button or in front of HMD/camera with a keyboard shortcut.

Implementation

In GodotWestonCompositor, the record field

_gwcSurfaces
  :: GodotWestonCompositor
  -> TVar (M.Map WestonSurface GodotWestonSurfaceSprite)

becomes

_gwcRunningApps
  :: GodotWestonCompositor
  -> TVar (M.Map WestonSurface App)

App

An App type would contain a Card as well as data on the application. The data is parsed from the .desktop file and at least includes icon, launch command, path, and description.

Card

A Card is a registered class extending GodotMeshInstance and would be the equivalent of the current GodotWestonSurfaceSprite. It would have a card-shaped mesh with a UV-mapped shader to display (depending on state) the WestonSurfaceTexture, application name, window title and maybe some card body text (whatever it might be - desc while unopened, notifications in minimized state?).

[KaneTW]

Idea seems sound. Are there any existing VR games/apps with a card deck to borrow ideas from?

[georgewsinger]

Agreed: this idea sounds interesting.

Is the analogy that a card is to a Simula Application as a tray icon is to a Windows Application? For examples of tray icons, see the bottom of this photo:

With this idea, instead of having tray icons confined to the minimized application bar, they could float in space?

[lboklin]

Are there any existing VR games/apps with a card deck to borrow ideas from?

I don't have any direct inspiration off the top of my head that uses cards as such and I can't find much in the way of actual card-like VR UI demos, but I remember being impressed by this demo, and cards would allow a similar UX.

A few VR card games seem to be in development or early access etc but aside from having cards and maybe showing a hand of cards next to your controller I haven't seen any noteworthy elements relevant to us.

Is the analogy that a card is to a Simula Application as a tray icon is to a Windows Application?

A card in your hand might be the equivalent of a minimized window in a traditional taskbar (like in Plasma, Gnome Shell or Windows like you see in your screenshot to the left of the tray thing) or one of the entries shown when switching between apps on Android.

A card in your deck (application not running) would traditionally be an entry in your application launcher or app drawer on Android.

When an application is running and open it wouldn't look much like a card as all the surface is used for the window's contents.