Comments on the Plan for the big upcoming Scenic update

[boydm]

I'm working on a big update to Scenic. lots of change, especially at the Driver level.

I'm just to put up an overview in the wiki portion of the repo.

Please add comments to it here...

[crertel]

Re: 3x2 matrices

I triple-checked my work on the matrix stuff...if we're looking to support perspective transforms (for pseudo-3D effects), we might need full 3x3 ( see here, here, and here). Mea culpa, I screwed up. Affine transforms map parallel lines to parallel lines, but for pseudo-3D perspective stuff (say, mapping a rectangle to a trapezoid) we have to go up a level to projective transforms (nice writeup here). In short, we figured out what that extra column was for (block diagram here for 3D, but just shave off the Z coordinates and the 3x3 matrix has the same general structure).

tl,dr: We want 3x3 matrices unless we want to make fake 3D effects (like in #208) impossible. It might be possible/preferable for performance reasons to limit to affine transformations, and then let the end-user figure out the transform work...but I don't know how much transforms are pushed down into viewports. Would be nice if kept the option open with a slightly more complicated viewport model (e.g., affine transform 3x2 type plus a general transform 3x3 type).

Re: input

• Are we caching transform matrices and their inverses in the graph? If not, would that provide a bit of a speed boost?
• I like the input: true idea...would that need to be at the lowest level (shapes), or just containing groups? Would some kind of broadphase collision detection (say, gross bounding boxes) be a good first pass to help narrow down who should receive inputs?
• You mentioned that a viewport needs to check every scene and every component...is there a notion of "active" scene that can be used to ignore input for scenes that don't matter?

Misc

Do the draw API solid(xy) and hole(x,y) calls need arguments at all?

[Eiji7]

@crertel Sorry as I'm not really good in this stuff … Can't we simply have 3x2 matrices in scenic core and just support to pass any other matrices? That way third party library would basically copy-paste old scenic matrices 3x3 and use them in their helpful functions for performing 3D effects?

Also just for sure … What do you mean by "fake 3D"? Do you mean workaround for flip using scale (like in linked issue) or just that we would have 3D-like flip effect drawn on 2D scene? i.e. Which effect would see end-user flip or scale simulating flip?

[boydm]

@cretel the arguments on solid() and hole() are probably wrong. I haven't done those yet...

[crertel]

@Eiji7 I mean "fake 3D" in the sense that vectors are all elements of R2 (e.g., 2D space), but on screen a transformation can be applied to make it look 3D--in particular, the linked perspective transforms that would let you have foreshortening. Like, if you held up a phone in front of you, turned it sideways, and then rotated it around the vertical axis, you'll notice that the edge of the phone closer to you appears larger than the edge farther away from you. For good 3D flipping animations, you can't just flip and scale horizontally, you need to also have that perspective element.

[vacarsu]

It would be really convenient to have an easy way to get a components transforms in global space. This would make it much easier for example to render tooltips under buttons in a higher order component to keep them rendering above other components in the scene.

[boydm]

It would be really convenient to have an easy way to get a components transforms in global space. This would make it much easier for example to render tooltips under buttons in a higher order component to keep them rendering above other components in the scene.

Interesting. Are you thinking some sort of API where you would provide a graph and an :id, you would receive it's contextual matrix? What context would that matrix be in? The root scene's context? If this is a component would it be in that component's context?

Also, in order for this to work, the scene you are querying would already need to be pushed to the viewport. Just supplying a graph by itself isn't enough context as the relative matrix is built up from the scenes above it that reference it.

[vacarsu]

an API where I provide a graph and an id could work. As long as I could pass in the root scene graph and find a deeply nested component, and get the matrix relative to the root scene. Essentially this sort of API would require me to store the root scene graph in a GenServer elsewhere, and keep it up to date to pass it into the API whenever I need the global space. Is my line of thinking correct here?

[boydm]

Maybe... I'm a bit confused which graph you want to add the tool-tip to. If you get the target in root coordinates, but add it to a nested component's graph, that will wrong. So the api is probably more like get_matrix( dst_graph, src_grph, src_id)

That sound right?

[boydm]

To make it more confusing, there is a master, global transform you can add above the root (on the viewport itself)! That is currently expressed in the config, though I'm exposing it via api on the viewport. This lets you do things like rotate the entire screen 90 degress or shrink it to fit a screen or whatever.

So even if the goal is to add the tooltip to the root, it is still a relative matrix.

[vacarsu]

So what I currently am doing as a workaround - I have a tooltip layer which I render on my root scene. When an Icon button is hovered I send a message to the tooltip layer with the tooltip component. I then get the cursor position using xdotool using a terminal command (I really dislike this way of doing it). This gives me a global position close to the button where I then render the tooltip at. This gives me something close to the desired affect. However, it would be so much better to be able to get the global position of the icon button, so I can render the tooltip perfectly where I want it on the tooltip layer.

[crertel]

Could we go ahead and cache the concatenated transform on each node that would get it all the way back to the root? And/or its inverse?

Don't we already do something like that?

[vacarsu]

The concatenated transform being cached and accessible from Primitive.get_transform() would be really convenient. I do this manually in one place in the application I'm building. I get the translate of a primitive and pass it up to the parent and add it to the parents translate. This works fine as long as your graph tree is relatively simple. It becomes an event dance once your structure becomes complicated, then moving a component in the tree becomes a whole new beast >.<. Reminds me a bit React prop drilling.

[boydm]

Update: This is a pretty major update. I think I'm on the last circle around with scenes & input. Part of the goal here is to get rid of all the "magic" parts and have it all be tracked really cleanly. Hard to achieve when incoming input needs to be translated to the appropriate coordinate space for the appropriate scene.

Should be much cleaner when this is done. Aiming to have this be much closer to the 1.0 api set.

[boydm]

Hey all. Question for you.

I'm in the "re-testing each component" phase of this beast, and I noticed something weird.

I always intended the {0,0} coordinate for each components coordinate space to be at the upper left hand corner of the component. In other words, if all you do is have a scene with just one component (such as a button) and no transforms at all, then that component would be fully visible and tucked into the upper left hand corner of the ViewPort.

Button, Dropdown, Slider, and TextField do this correctly.

Toggle is centered vertically, which means the upper half is above the top of the window. CheckBox looks like it is set vertically on the text baseline, which puts most of it above the window. RadioGroup is weirdly off. The top radio is set to the baseline like checkbox but the whole thing is a little too far to the left??

So... I'm temped to "fix" these by having them all behave the same as Button/Dropdown/Slider/TextField. BUT, this will move things around a little in any existing scenes, causing some manual re-positioning.

What do you think of that? Too much? I'd like to get it right now, before it gets pushed into lots and lots of new devices.

[vacarsu]

I think might as well fix it now. I don't think I'm actually using any checkboxes or radio toggles oddly enough. All components should render equally position wise.

[axelson]

I obviously can't speak for everyone, but from my point of view I think this is a good time to fix alignment issues like that. As far as I'm aware mainly @vacarsu and @JediLuke have built much substantial on top of scenic, and I don't think either would be considered currently "in production".

FYI, I cross-posted this to the Elixir Slack

[boydm]

Thanks you @axelson for posting to the slack! That is a tool I just rarely use...

[cortfritz]

Can’t you have a backcompat flag to preserve the previous offsets? Anybody wanting to upgrade can just set that flag.

Sent from my iPhone

On May 15, 2021, at 6:53 PM, Boyd Multerer @.***> wrote:

 Thanks you @axelson for posting to the slack! That is a tool I just rarely use...

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[boydm]

@cortfritz Cort! Why am I not surprised you would ask that?

Yeah maybe? But it would be a bunch of work. Was partly wondering if that would get asked.

[crertel]

It's pre-1.0 software; I wouldn't bother with a compat flag.

This seems like a good way of cleaning up the sort of inconsistency that'll inevitably cause headaches down the road.

[jasonmj]

Another vote for cleanup and consistency here.

[boydm]

Next one. I want to change the :text_height style from a hardcoded number of "pixels" to a percentage of the font size. i.e. instead of specifying 50, you would instead use 1.2 (which would then automatically scale as you adjust the font size). This is how typography works normally.

So... Should I just change the meaning of the existing :text_height style or deprecate it and create a new :line_spacing style? The former means less porting work. The latter is clearer as it would fail on :text_height at compile time.

What do you think?

[boydm]

The new spacing would use a sensible default of 1.2 if you don't specify anything...

[jasonmj]

Seems like a different functionality, so a new name would be justified. If it were :line_height it’d click immediately for folks coming from css, but :line_spacing is also quite clear.

[crertel]

@boydm can you go into that a bit more?

percentage of the font size.

Is this reference size based on the height of the font in pixels, based against the x-height, the ascender-height, or which one, exactly?

(Font height is tricky, see the leading diagram for different measurement points).

Could you do a quick diagram here with some measures to show what spacings you're thinking of?

[boydm]

@cretel I'm talking about baseline-to-baseline spacing between lines of text in a paragraph.

@jasonmj You are right. It should be called :line_spacing to be similar to css

[jasonmj]

To clarify, :line_height would be similar to css, rather than :line_spacing.

[boydm]

Yeah. I'm going with :line_height

[crertel]

Sounds good to me!

[boydm]

Scenic and scenic_driver_glfw now have v0.11 branches you can look at

https://github.com/ScenicUI/example is a working example that uses them.

I'm planning to move scenic and related repos to the ScenicUI group

[boydm]

The docs are largely out of data as that is the next big pass I need to do.

So instead, start by reading this: https://github.com/boydm/scenic/blob/v0.11/guides/upgrading_to_v0.11.md

[boydm]

v0.11.0-beta.0 is published to hex. I'll close this issue when the RC goes out.