[feature] match multiple windows & configuration file

[slickplaid]

I like to put my home security cameras in view for some games. I like to swap profiles between action games where I need the whole screen and only a select few camera views on my wrist, to another view that loads them all above eye level so I can glance up to view them.

I've found that streaming them over rtsp via VLC and then using graphics capture on those suffers very little performance hit compared to a webpage with all the cameras on the single browser window. This results in 12 small VLC overlays vs 1 browser overlay.

To accomplish this, I have a shortcut which opens all 12 VLC instances, waits for them to load, and then loads steam VR, which subsequently loads Desktop+.

All 12 VLC instances are saved as 12 unique "Overlay" and saved as a multi-overlay profile.

Problem

Whatever is being matched against to reload and persist an overlay is not consistent or has a missed state (null or undefined) which can break the overlay until re-selected in settings.

1. When I switch profiles, reload the game after a reboot, or switch to a profile where the VLC windows are not saved, frequently the remembered vlc windows will fail to load, or load something else entirely.
2. The app does not allow remembering of an app when the configuration is saved for an unrelated change, if the app isn't currently matched with an app running.

It also seems like something in the configuration saving fails to save something completely or it gets mixed up when swapping between profiles, especially if I change anything else in the settings. Sometimes swapping profiles works perfectly. Sometimes it does not work at all and forgets every match between a window on the desktop and the overlay it was set to.

This is especially noticeable if a window wasn't present or wasn't matched when saving, even if you weren't adjusting the overlay and what it captured. It appears it overwrites matches first before saving resulting in a "nothing selected" match.

Solution suggestions

The VLC window names are rtsp://(ip):(port)/(uid-for-camera) - VLC media player and should not change between close and reload. 80% of the time it works every time but once it starts failing to find the cameras, the ones missing permanently fail to load like the match on window title is no longer saved and never recover. It requires rematching the overlay with the window.

1. Allow saving of a configuration file in line with 12-factor apps and allow for the user set the window name. I do not see anywhere in the docs where the configuration is stored currently and doesn't appear to be user accessible.
2. Allow matching against a group of apps of similar name or supply a regex or some kind of rough pattern matching to match on window names.
3. Handle always returning the same first match (match -> sort -> return n[0])
4. or more ideally, handle matching against more than one app and display those in a grid as a single overlay.

Matching would require handling of multiple apps if you match more than one, so matching against a single app could (hopefully not randomly, but predictably) always select the first match OR allow you to select more than one of the match and display them as overlay n, n+1, etc

If I wanted all vscode windows, I could match against (.*) - Visual Studio Code. VLC, I could match against (.*) - VLC media player, or match against the specific ID for my camera /rtsp.*\/(abc321) - VLC media player.

If you wanted to get fancy with it, allowing a region of space that the matched windows could occupy with a setting for wrapping, number of columns/rows, etc. to handle an indeterminate-until-running number of overlays.

[elvissteinjr]

Hi,

from the sounds of you loading multi-overlay profiles manually, you're using the current stable branch version, I assume?

While it doesn't address all your concerns, window capture handling has been improved quite a bit on the NewUI Preview branch. In particular, window captures are automatically restored when the window re-appears and you can enable strict window matching to only get 1:1 title matches picked up for each overlay. Application profiles can also automate loading the right profile for each VR app. Profile loading should also be more robust, though the main issue on the stable branch is the lack of active window monitoring.

So while not feature complete yet, I highly recommend trying the NewUI Preview builds out for this. Should handle this scenario better. But if my main assumption is wrong here and you're already using a NewUI Preview build we'd have quite some odd behavior there that needs investigating.

Configuration is stored in the application directory, with the main config being config.ini and profiles being in the profiles directory. Plain text and user-editable (changes to the main config will get overwritten if done while Destkop+ is running, however). While writing to the application directory is debatable (and don't fit your twelve-factors), I went with the intent of having a self contained app that does not write to other locations on the system. It's true that the documentation doesn't really call those files out, though.

As for the suggestions, the current fuzzy matching method checks for executable name (NewUI includes window class as well) + title, with the title being stripped off a space-separated part on each iteration with match until the last " - ". Intent here is still match with changing document names, but avoid other windows belonging to the same application. NewUI branch has the option to turn on strict matching as mentioned before. Window handle isn't considered here as it's not reliable after the window disappeared. It may even get re-used for an entirely different application.

I feel like this should be sufficient for most uses. Adding additional regex pattern matching would be possible, but seems kind of niche considering it's very unwieldy do set up in VR. Having multiple capture targets on one overlay is a no from me. This would break some assumptions throughout the application's code and prevent some optimizations for little benefit.

For the other things it's really a question of how often your layout really changes and how many people will make use of more complex configuration options like this? Since you seem to already have your own launch logic for this setup, I believe you could incorporate explicit titles to your windows to have matching always work and capture loss setting to "Hide Overlay" to not see the overlays when the windows aren't there (both assuming NewUI branch).

[slickplaid]

you're using the current stable branch version, I assume?

correct, up until yesterday

window capture handling has been improved quite a bit on the NewUI Preview branch.

It's so much better than before. Before it was a chore to update all of those windows. Now I can just pick the ones I want and add the overlay without much thought. So much has improved. Kudos.

window captures are automatically restored when the window re-appears

This is a vast improvement and nearly fixes all of my issues or workarounds in the previous ask.

Application profiles can also automate loading the right profile for each VR app

also awesome and I've been using it extensively

Profile loading should also be more robust

100% and noticeable improvement

I highly recommend trying the NewUI Preview builds out for this. Should handle this scenario better.

It does and I really like the changes. It handles what I was asking in better ways by making the ease of creating a group of overlays so simple. By group, I mean just all of the windows I had opened saved into a profile. Your release does exactly that and without any problems that I've found so far.

Configuration is stored in the application directory, with the main config being config.ini and profiles being in the profiles directory. Plain text and user-editable (changes to the main config will get overwritten if done while Destkop+ is running, however).

This is fine and appears to work well. I dove into the application code with the intention of creating a PR before I found your beta branch.

While writing to the application directory is debatable (and don't fit your twelve-factors), I went with the intent of having a self contained app that does not write to other locations on the system.

Granted the 12-factor apps are primarily focused on deployable/server concerns especially with environment scope/variables, I think including any configuration is better than most. I don't write much user distributed code, so there could be some windows concerns (e.g. roaming profiles, unknown setups that use user data different) that might come in to play with the ease of that decision since you already know the app directory is writable and various distribution methods (e.g. steam) that could cause complications. I don't fault it.

It's true that the documentation doesn't really call those files out, though.

If I get some spare time I can take a stab at a PR with some suggestions for documentation.

As for the suggestions

I think you've addressed my concerns with the beta. I'm excited to see future changes.

Having multiple capture targets on one overlay is a no from me. This would break some assumptions throughout the application's code and prevent some optimizations for little benefit.

Completely understand.

how often your layout really changes and how many people will make use of more complex configuration options like this?

The goal is for it to not change often unless game configuration requires it. The ease of being able to switch applications, pop the focused app out, and very quickly and easily fixes nearly every problem.

Since you seem to already have your own launch logic for this setup, I believe you could incorporate explicit titles to your windows to have matching always work and capture loss setting to "Hide Overlay" to not see the overlays when the windows aren't there (both assuming NewUI branch).

I'll check this out, but for now I'm more than happy. Thank you!

The only issue I've had with the new branch is the default behavior of the keyboard displaying at inopportune times. If you hover or click accidentally on a window that accepts input, if an input field autofocuses with that input, keyboard is displayed but luckily it was toggleable and easily fixed and gratefully that behavior seems to be retained in the profile and app settings which is awesome.

Great release, and thank you for the detailed response to my criticisms and concerns.