Add support for transparent HTML overlays

[Merudo]

Is your feature request related to a problem? Please describe.

Transparent HTML overlays would be a great way to display UI elements, such as a health meter, clickable icons that can launch macros, and a picture of the selected character.

A good example is provided here by @melek : https://forums.rptools.net/viewtopic.php?f=26&p=275748

Describe the solution you'd like

One option would be to extend the dialog() function to allow for invisible dialogs, which could then function as overlays

Another would be to make a whole new function for overlays.

Additional context

The implementation of overlays poses two difficulties:

1) the user should be able to specify where they want the overlay to appear on the map (center, corner, side, etc), and the overlay should stay on the same relative spot after resizing the map or docking new frame 2) the user should presumably have some way to close the overlay manually. Maybe some sort of close button could appear on overlay after a mouse over?

[Merudo]

For #1 I'm thinking we could add parameters to dialog.

Parameter overlay. By default 0, setting it to 1 would make the dialog invisible.

Parameter relativePosition. This parameter could work for both overlay and non-overlay dialogs. The nine allowed options would be center, top, bottom, left, right, top-left, top-right, bottom-left, and bottom-right. For overlays this would be where they are displayed relative to the map, and for non-overlays it would be relative to the MapTool overall frame.

Parameters x and y. These parameters can give an additional offset over the relativePosition parameter. So y = 100 and relativePosition=bottom would make the dialog show up 100 pixel above the bottom.

[JamzTheMan]

Could we utilize the current "docking" and add a button that removes the window decorations (and allow for transparent backgrounds?

And maybe under Window menu -> Custom (or better name?) we show all the current window dialog frames and allow for the window decorations to toggle and or close the window?

[Merudo]

@JamzTheMan I assume the reason for using docking frames is to let the user move their UI overlay elements as they wish using the mouse?

[JamzTheMan]

Yes, that and it would always center, snap and fill automatically (and reload on campaign load), Although not sure about "click through" to the map will be a problem or not? You get some built in stuff, although, admittedly it couples the functionality to the existing docking framework which we would eventually get rid of.

I'm ok with your suggestion as well, just tossing out ideas and seeing what sticks.

[Merudo]

Yes, that and it would always center, snap and fill automatically (and reload on campaign load),

Seems to me it would be helpful to implement the relativePosition , x and y parameters I mentioned. That way the overlays would appear at the correct spot, instead of all bunched up in the center.

Although not sure about "click through" to the map will be a problem or not? You get some built in stuff, although, admittedly it couples the functionality to the existing docking framework which we would eventually get rid of.

I'm ok with your suggestion as well, just tossing out ideas and seeing what sticks.

I think I've figured how to pass mouse events to the zoneRenderer so that it becomes possible to "click through" to the map underneath. Hopefully there won't be any issues.

I like the idea of the user reorganizing their UI as they wish by moving the overlays around.

If the overlay is movable, how should the overlay behave if the map is resized, for example if a frame / window gets docked or undocked? If a frame is docked and the overlay doesn't move, the overlay may become inaccessible. So presumably, we would want to "anchor" the overlay to the closest corner / side once placed by the user, so that it remains proportionally at the same spot after a resize.

[JamzTheMan]

Ya, I would think having a "corner" overlay to be very useful.

I assume all coordinate/offset x,y, etc would be relative to the "map" frame and NOT the whole UI? Then you shouldn't have to worry about all the other windows.

Should this be it's own "function" instead of extending the dialog5? (I like the name overlay). I can't think of a usecase were I would need a transparent overlay (vs a normal frame/dialog) that wasn't over the map?

Also, what do we do with Layers dialog for the GM? Is it always over the overlay? (drawing panel as well although that makes sense to pop it over overlays when needed)

[melek]

Exciting stuff.

I agree that in macro script this could be a separate overlay() function as Jamz is suggesting, that makes sense to me! Edit: Agreed, these would be relative to the mapview frame and could not appear outside of it.

The Layers/Drawing/Adjust Map modals should simply render directly over any Custom UI overlays as if they are a part of the map. Any currently open modal should have a corresponding class applied to overlays for CSS to respond to - I feel like the current UI should be generally 'first class' and user-defined overlays should be given information to work around it if they choose.

Eventually (once html5 is applied to them, probably), I think overlays should be responsive based on the width and height of the Mapview with CSS media queries and other classes assigned to the overlay body.

I am still interested in having UI defined in the campaign properties and have the initial layout at least be strictly defined for a campaign; each overlay needs HTML and CSS and possibly Javascript and images, so it makes sense to make each overlay a tidy package in Campaign properties. A compact edit UI like https://jsfiddle.net/ would make a lot of sense, though that seems a bit ambitious - maybe down the road? :)

Anyway, 'Custom UI' tab in Campaign Properties that lets you create a list of named overlays would be a little more approachable for GMs and elevate the feature a bit as a way for GMs to make their frameworks distinctive. I worry if not, they will only be used by a minority of more advanced users. (This might dovetail into #1422 and the overlays might be a folder in a campaign notebook I'm not sure).

In my dream world where players can connect on their android and iOS devices, mature frameworks can have a custom UI defined that makes playing on a tablet a breeze.

[melek]

I just had a thought; if click-through does end up working while still allowing button presses, would custom overlays being combined into a single HTML 5 overlay that is the entire width/height of the map view which combines all overlays as absolutely positioned DIVs be more performant ?

I just had a thought of framework creators adding small, individual overlays for lots of pieces of overlapping UI creating a performance issue (like chrome with lots and lots of tabs), and smushing all defined overlays into a single HTML document might be a lot faster.

edit: brain switched half a sentence around..

[JamzTheMan]

Ok, just had another shower thought....

@Merudo How about adding a visible attribute as well with default true but also allow on-hover, on-select (and false?)

This may run into/over/compliment/replace #1424 as you could now create "stat sheets" of any sort where ever you wanted.

Another useful "coordinate" would be an offset based on a token for on-hover and on-select. Envision a round token getting a nice radial "HUD" if you will around/over a selected token! How cool would that look? It would/could replace/enhance health bars & states by letting us take them off the token and using them in an overlay over and around the tokens.

[melek]

Created a brief fiddle demonstrating a sample single HTML document overlay system with 9 potential overlay base positions:

https://jsfiddle.net/digiacom/dvu1h4bo/

[JamzTheMan]

Can we position said overlay relative to another overlay? Lets say I had 1-4 overlays I wanted to display in the upper right, each with 0-n padding between them. And lets say I may want to display 1, 2, & 4 for player 1 vs 1, 3, & 4 for GM, I wouldn't want that gap in there. Yes, this could be 1 overlay but lets entertain I need 4?

Or better yet, lets say I had 0-n overlays (maybe each is a "state") and as such I want them to start in upper left but "wrap" down into a second row?

Should we instead treat the map "frame" as one large "window" that we can pass CSS to and organize the overlays that way? Does each "overlay" need to be it's own "dialog/frame"?

Also are these elements in the overlay intractable/clickable (I assume so and the click-thru is only for transparent parts of the overlay?)

[melek]

@JamzTheMan At least in my approach we could structure an overlay to be another overlay's parent, then style that overlay to wrap a flexbox container around the contents if that is happening. By styling the parent div's options we could change the flex properties, but columns would be a good default.

Updated the fiddle with an example, try clicking or manually adding and removing copies of #OVERLAY-004 & 005: https://jsfiddle.net/digiacom/dvu1h4bo/

[JamzTheMan]

Yea, I'm just wondering if it would be best to have a single overlay acting like a "glass" layer the framework launching several overlay windows each running it's own webengine. A single instance of a webengine to render a whole page of elements may be more performant.

Although I think we are saying the same thing really. Develop all of your "overlays" in a single html/css "dialog" and forget about having to launch multiple "overlay" dialogs each with there own relativePosition.

[Azhrei]

(This has been exciting stuff to watch develop. :))

My only concern with using CSS is how touchy CSS can be about layout issues. Is there an option to create some kind of LayoutManager and use that? I could see a LayoutManager that gives each child a priority and a position, then lays them out in priority order. The position might be "+5+0" to mean 5 units to the right and 0 units up from the previous component, and "-10+10" would mean 10 units left and 10 units up, always relative to the corner specified by the offset. So numbers that are +,+ would be from the lower left corner and numbers that are -,+ would be from the bottom right corner. This is the style used by the X Window system to place child windows...

Whatever comes out of this, I'm looking forward to it. ;)

[JamzTheMan]

(This has been exciting stuff to watch develop. :))

My only concern with using CSS is how touchy CSS can be about layout issues. Is there an option to create some kind of LayoutManager and use that? I could see a LayoutManager that gives each child a priority and a position, then lays them out in priority order. The position might be "+5+0" to mean 5 units to the right and 0 units up from the previous component, and "-10+10" would mean 10 units left and 10 units up, always relative to the corner specified by the offset. So numbers that are +,+ would be from the lower left corner and numbers that are -,+ would be from the bottom right corner. This is the style used by the X Window system to place child windows...

Whatever comes out of this, I'm looking forward to it. ;)

FWIW I'm not suggesting to use CSS to layout swing dialog aka "windows" but rather there is a full width/height invisible pane that is always on and overlaying the map and the user can via macro add content to it. If they wanted a "status panel" in the upper right corner, you would use standard HTML/CSS to place it there.

I would think you would be able to layout some simple elements in the corners/sides this way but ya sure, YMMV, i mean, it IS webkit after all so /shrug

[Azhrei]

Should we instead treat the map "frame" as one large "window" that we can pass CSS to and organize the overlays that way? Does each "overlay" need to be it's own "dialog/frame"?

That's the question. If there is a single overlayPane (that sits between the contentPane and glassPane?) and it's a single WebView component, the CSS is being used to position the individual overlays within the WebView and thus, within the overlayPane.

But if that overlayPane holds multiple WebView components, then it needs some kind of layout manager to position them (something like SpringLayoutManager). I was suggesting that if this were the solution chosen, the X Window system's "geometry" syntax might be useful in coding the layout manager.

It's kind of a toss-up. Using a single WebView means CSS has to be used and CSS is finicky, particularly when it comes to rule specificity. But LayoutManagers are not exactly dream components either and they can be hard to get right. And the CSS rules are already written...

[melek]

I like the idea of a single HTML overlay since it gives a very high ceiling of possibility immediately. This is assuming the click-through method Merudo is working on works as expected and can 'catch' clicks/hover events that the overlay wants to react to (anchors, buttons, inputs; elements with onclick handlers and hover effects).

As for CSS being finicky, since we can rely on exactly the HTML rendering shipping with MapTool, I feel like CSS will be very reliable and will render the same across all clients. We can scope CSS for a given overlay to that overlay's ID, like I do in my fiddle example.

I like the Window X geometry you propose; I think that we can use that system also in CSS using the top, left, bottom, and right attributes - so for an overlay we could store the offset in a Window-X friendly way and it would work fine. At very worst, the overlay could be stored with offsets that need to have some work done to them before they're used in CSS.

To get WAY ahead of myself, this is an ambitious mockup of the kind of UI I'm imagining in Campaign Properties. This is extra ambitious because it implies that MapTool would additionally ship and work with Vuejs, though I don't really expect that.. ;)

The overlay class would be rendered in the order of the overlays in the list; all the HTML, CSS and Javascript would be compiled and cacheable. I only used a Vue pattern because I just am not sure how to 'watch' a variable in Javascript with MapTool to update an HTML frame, but I hope the ideas get across.

[cwisniew]

I like the idea of a single HTML overlay since it gives a very high ceiling of possibility immediately. This is assuming the click-through method Merudo is working on works as expected and can 'catch' clicks/hover events that the overlay wants to react to (anchors, buttons, inputs; elements with onclick handlers and hover effects).

I second this, its both easier to implement and more flexible.

As for CSS being finicky, since we can rely on exactly the HTML rendering shipping with MapTool, I feel like CSS will be very reliable and will render the same across all clients. We can scope CSS for a given overlay to that overlay's ID, like I do in my fiddle example.

I like the Window X geometry you propose; I think that we can use that system also in CSS using the top, left, bottom, and right attributes - so for an overlay we could store the offset in a Window-X friendly way and it would work fine. At very worst, the overlay could be stored with offsets that need to have some work done to them before they're used in CSS.

This above. Plus we open the pool to potential contributors for UIs so much.

[Merudo]

That's the question. If there is a single overlayPane (that sits between the contentPane and glassPane?)

Right now I'm using a LayeredPane, but adding stuff to the glassPane is another option.

Should we instead treat the map "frame" as one large "window" that we can pass CSS to and organize the overlays that way? Does each "overlay" need to be it's own "dialog/frame"?

That's the question. If there is a single overlayPane (that sits between the contentPane and glassPane?) and it's a single WebView component, the CSS is being used to position the individual overlays within the WebView and thus, within the overlayPane.

...

It's kind of a toss-up. Using a single WebView means CSS has to be used and CSS is finicky, particularly when it comes to rule specificity. But LayoutManagers are not exactly dream components either and they can be hard to get right. And the CSS rules are already written...

One advantage of multiple overlay components may be faster updates ; if a single overlay needs updating, it will in general be faster to update that single component instead of processing the html for the entire overlay screen.

However, I do agree that having one single overlay html component taking the entire screen is probably more effective. For one, it then becomes possible to arrange the layout entirely with CSS & javascript, which is preferable to us reinventing an API. I'm especially excited about the draggable option in HTML 5 - it could allow the player to easily rearrange the placement of overlays to their liking, for example.

But if that overlayPane holds multiple WebView components, then it needs some kind of layout manager to position them (something like SpringLayoutManager). I was suggesting that if this were the solution chosen, the X Window system's "geometry" syntax might be useful in coding the layout manager.

I made a basic LayoutManager for JLayeredPane to support multiple component. Each component is given a halign (left/center/right), a valign (top/center/bottom), and a xOffset and yOffset. So for example specifying right + top with a xOffset=-40 offset will draw the component 40 pixels away from component at the top right of the panel, with 40 pixels between the left side of the component and the right edge of the screen. It works good enough, but having it done by CSS / JS would be more flexible.

[melek]

Are Ajax techniques possible in Java web components so we can dynamically update the dom without forcing a total refresh? Finding a way to provide updates without a total reload of the html would be great for snappy performance - but I just don't know if something like Vue/react can work in this context...

[Merudo]

PR #1438 implements the feature for html 3.2. Will post an example soon.

[Merudo]

Are Ajax techniques possible in Java web components so we can dynamically update the dom without forcing a total refresh? Finding a way to provide updates without a refresh would be neat

It's possible with HTML 5 in WebView, see #1362. Unfortunately WebView is JavaFX, and to my knowledge it is not possible to have satisfactory transparency when using JavaFX components in Swing. So this approach won't work for overlays until we switch to JavaFX.

[Merudo]

Another downside of a single HTML overlay: in HTML 3.2, it is not possible to vertically align an element to the bottom of the page. Having an overlay element snap to the bottom border or the vertical center is therefore not possible.

One option is to have three HTML overlays: one aligned to the top, one aligned to the bottom, and one aligned to the center.

[JamzTheMan]

So the question is, do we move forward with an html3 solution and completely redo it in html5 later or...?

[cwisniew]

The problem with a HTML 3.2 solution is we are adding more baggage that is hard to be removed while I want to actually start removing baggage.

The problem is we get to the point of being able to have a HTML5 solution having this in place will delay it greatly because we get to the point where we can't implement it due to fear of breaking everything out there.

This is also why I don't want to add JS macros in frames willy nilly

[melek]

@Merudo This was really easy to setup! The only 'gotcha' I had with this first attempt was that frame events don't update the overlays.

That said, If it would make it harder to get overlays and slow down development by adding more legacy features, I think I would haltingly vote to wait on developing this until JFX is in. That will also be a nice 'killer feature' to advertise with that new release along with all the other HTML 5 / JS excitement.

That said, I am very interested in doing whatever I can help to develop that JFX version of MapTool - if there is an issue about that already, or if I can learn a specific skill to help port UI, I'm interested in learning... I want this long talked about JFX transition pretty badly after seeing @Merudo's feature mockup :)

[Merudo]

PR #1439 adds support for frame events. Thanks @melek for the heads up.

[Phergus]

@Merudo Three things:

1. Can't use the Level selector to change active layer
2. Can't use the drawing tools palette.
3. No way to turn off the overlay once it's up.

[Merudo]

Thanks @Phergus ! PR #1440 fixes these bugs.

[Merudo]

Next overlay version HTML5. One things is left to fix: clicking on the overlay buttons and hyperlinks works, but also sends a click to the map.

[Merudo]

PR #1485 fixes the last problem and blocks clicks on hyperlinks, input elements and buttons from going to the map.

I also added the ability to selected which overlay elements should block clicks to the map by adding the custom css style --pointermap. If this style is set to block, clicking on the element will not affect the map. So, for example

<img src="mypicture" style="--pointermap:pass">

will let clicks on the picture affect the map, while

<img src="mypicture" style="--pointermap:block">

will block them.

Hence this way the overlay can have rain effects (should not block clicks to the map), as well as character pictures (should probably block clicks).

[Merudo]

Here is an example, reusing @melek's overlay pictures:

The bottom UI part acts like a footer, and appears at the bottom of the screen even after resizing the map.

It was created by this simple MapTool script:

[h: topImage = tableImage("UI", 1)]
[h: bottomImage = tableImage("UI", 2)]

[overlay():{
<style>[r: '
.footer {
  position: fixed;
  bottom: 0;
  text-align: center;
  margin: 0 auto;
  width:100%;
}']
</style>
<img src="[r: topImage]">
<div class="footer"><img src="[r: bottomImage]"></div>
}]

[Merudo]

PR #1534 implements multiple overlays. I still haven't added the overlay menu in the UI but otherwise it is shaping up to look pretty good.

[Phergus]

re: HTMLFrameFactory.java

        } else if (keyLC.equals("zorder")) {
          try {
            zOrder = Integer.parseInt(value);
          } catch (NumberFormatException e) {
            // Ignoring the value; shouldn't we warn the user?
          }
        } else if (keyLC.equals("title")) {

Yeah. We'll need to report non-numeric zorder to the user and probably mention using default (or next) z-order.

Is a function to set/change zorder needed? Or just call the overlay() function again with a new zorder?

[Merudo]

PR #1543 adds an overlay menu where each overlay can be turned on or off.

A bare-bone demo. Click on the macros of the Campaign panel to enable the three overlays:

OverlayDemo2.cmpgn.zip

[Merudo]

re: HTMLFrameFactory.java

        } else if (keyLC.equals("zorder")) {
          try {
            zOrder = Integer.parseInt(value);
          } catch (NumberFormatException e) {
            // Ignoring the value; shouldn't we warn the user?
          }
        } else if (keyLC.equals("title")) {

Yeah. We'll need to report non-numeric zorder to the user and probably mention using default (or next) z-order.

We could use the error message macro.function.general.argumentKeyTypeI:

Argument key "zorder" to function "overlay" must be an integer.

Or, should a new error message be written?

Is a function to set/change zorder needed? Or just call the overlay() function again with a new zorder?

Currently, the overlay() function can just be called with a new zOrder.

[JamzTheMan]

I like it!

[Phergus]

We could use the error message macro.function.general.argumentKeyTypeI:

Argument key "zorder" to function "overlay" must be an integer.

That works for me.

Currently, the overlay() function can just be called with a new zOrder.

Good enough.

[Phergus]

Feature complete?

[Merudo]

I think so. Might be ready for an alpha / beta.

[Phergus]

Tested with examples provided. No issues seen.

[Phergus]

Page added to wiki. https://lmwcs.com/rptools/wiki/overlay_(roll_option)