Sticky Layers

[mal]

Problem:

There's currently (so far as I am aware) no good way to apply a single transform to a selection of displayables while having them remain available to attribute changes. i.e. applying a single shader to multiple layeredimage characters and being able to alter their attributes while the shader is running.

Tentative solution:

One way this could be done would be to place the layered images on a separate layer, and apply the transform to the layer.

Nested problem:

Unfortunately however attribute changes and show statements always use a tag's default layer irrespective of where it is currently being shown. This means that every show statement must include an onlayer clause, and that say attributes are non-functional unless the tag is on its default layer.

Nested solution: Sticky layers!

A proposed flag (config.sticky_layers?) which would result in the SceneLists instance storing the layer each shown tag is placed on, and the default_layer function preferring this map to the "true" defaults. The outward facing result would be that the initial show would still require the onlayer clause, but any subsequent show statement or say attribute use would receive the "correct" layer automatically, until that tag was hidden, at which point the SceneLists instance would discard the tag's entry, and the default_layer function (now encountering a "miss") would revert back to the "classic" method of determining the default layer.

---

I would note that needing to create and manage a separate layer for the less than 1% of scenes in a game that need this sort of thing isn't really ideal, so a better solution or options around group transforms - the root problem - would be most welcome, however in the interim the "sticky layers" functionality would help with working around that limitation, and seems to my mind a potentially valid and useful feature in it's own right, hence this write up.

---

Happy to take a crack at this if it's felt to be a valid feature request and worth pursuing. :)

[morbil]

Transform properties are inherited by subsequence showing of a specific tag. That includes shaders. Here's an example using assets from "The Question" from the SDK:

define s = Character("Sylvie")
define g = Character("Gylvie")

transform signalwarp:
    shader "3pood.signalwarp"

label main_menu:
    return

label start:
    scene bg uni:
        xysize (1920, 1080)
    show sylvie blue normal at left
    show gylvie green normal at right

    s "You've created a new Ren'Py game."

    show sylvie blue normal at signalwarp
    show gylvie green surprised at signalwarp

    s "You've created a new Ren'Py game."

    show sylvie blue smile at right
    g "Once you add a story, pictures, and music, you can release it to the world!"
    show gylvie green giggle at left
    pause

    return

And here's how that looks:

https://user-images.githubusercontent.com/48863282/206885121-fa18cc78-0fc7-4f85-9f07-f6d0fa798f03.mp4

As you can see you can change both attributes as well as apply other transforms and have them inherit whatever shader is applied

[mal]

Thanks for the suggestion. :)

Unfortunately, if you look carefully at your example when the sprites move on top of each other, you can see that an instance (for lack of better terminology) of the shader is actually being applied to them both independently, and relative to their bounds. This is most apparent when looking just to the right of the lamppost behind them, where the edge of the shader as it is applied to the blue girl cuts off, but the green girl's shader continues left.

This is different to the problem specified, which is to have a single shader acting on both, and relative to the screen (or other fixed point). On the technical side that would mean a single shader receiving an input frame containing both sprites.

[Gouvernathor]

So, in a nutshell, you want to apply a single transform to a set of displayables, with layeredimages among them, but not all the displayables being displayed ?

The "sticky" solution, as far as I understand it, would probably cause problems when showing screens having the same name as a defined image, and when showing the same image (tag) twice on different layers. Why not do your initial solution of a new separate layer, and use config.tag_layer ? It would require to manually enter the layer name for each tag you use, but at least it's not once per show statement.

[mal]

A somewhat pathological use case for this would be characters standing around rotating projector that casts various shapes in light into the surroundings. Say it's rotating clockwise, and the shapes are circle, square, triangle, diamond, heart, repeating. It wouldn't make sense to apply the shader to each person individually, as the viewer would see a circle pass over all of them at the same time, then a square, etc; when really the desire would be for the shapes to be applied relative to their position, such that a character on the right would be having a circle pass over them, while a character on the left was being affected by the diamond, and moving the sprite would result in them being affected by the relevant shape for that new position.

The config.tag_layer map isn't capable of this as config settings are not meant to be altered outside of the init stage, and the result of doing so means the information is forgotten when the game is restarted, and carried through to new games when not restarting - tl;dr: it's very bad time.

The potential issues with screens and backwards compat with games that may use the same tag on multiple layers is why this is only proposed as an opt-in via config.

[Gouvernathor]

The opt-in is a good thing, what I think is not is the fact that if you choose to opt-in to use the sticky version, you opt out of having the same tag on several layers and having screens named like some images. It shouldn't be an either-or, ideally. And I also think that a non-compat config option which changes the way statements work could yield issues with creators, because a lot of edge cases have to be re-learned entirely. It would be fine if it introduced a free new feature, but not if it removes another one in exchange.

Also, why would you need to change the default layer of a tag during playthrough ? In your example, when there's no projector, they're standing in their separate unaffected layer, on top of master. When there is a projector, you just apply the transform to the layer. If you require more liberty than that solution allows you, maybe it's too much an edge case to be solved in the engine ? At least not using the "sticky" solution, in my opinion. Maybe allowing a store-variable version of tag_layer, like we did for sideimages default tag some time ago ? But we need to ascertain if this really deserves it.

[mal]

The opt-in is a good thing [...]

This is very much opinion-based, but the ability to have tags with the same name on different layers seldom feels useful, and more likely to result in accidentally showing a sprite twice if needing to shift it between layers. The issue of screens conflicting with images on the master layer exists, but I don't think it's a blocker in the way you appear to. So long as the config switch is documented well enough to avoid surprises I can think of no reason why it wouldn't be possible to live with that limitation (trading one for another).

Also, why would you need to change the default layer of a tag during playthrough ?

It's a cut-down example to illustrate a point about group transforms. The reason to switch layers is a knock-on from needing layers to provide that functionality. If one of the characters in the example moves to the camera side of the projector, then they need to be on a different layer to be unaffected by the shader, but all their attribute-related things need to keep working. So whether the layer behind or in-front is master (i.e. the default), they still need to be able to move during a play-through.

If you require more liberty than that solution allows you, maybe it's too much an edge case to be solved in the engine ?

I do think the sticky layer concept adds value to the engine outside of the original problem posed, for the same reasons mentioned in the original issue (needing onlayer in each show, breaking say-attribute functionality), which is why I wrote it up as a feature in its own right, but provided the wider problem context that lead me there (in an effort to give wider context in case it helped others propose better overall solutions).

Maybe allowing a store-variable version of tag_layer, like we did for sideimages default tag some time ago ?

This provides a tool to deal with the issue, but it doesn't really provide any direct value to anyone outside of creators on the much more advanced end of the spectrum. It would have to be manually managed and kept in sync with the SceneLists objects, as well as have an understanding of game.contexts changes. Getting that right is a potentially huge foot-gun even for creators that know what they're doing. Conversely the SceneLists object is literally designed to track this kind of information.

[Booplicate]

The issue of screens conflicting with images on the master layer exists

While this is not exactly relevant, before I was able to write some cursed code to show/hide screens via default show/hide, which can lead to undefined behaviour. RenPy already doesn't support (?) using the same names for screens and images (which is fair imo). So I don't think it can be used as a counter-argument for this feature.

[Gouvernathor]

Not exactly. It's currently not supported to have both an image and a screen with the same tag displayed on the same layer at the same time. But otherwise it's allowed. Only hide screen my and hide my onlayer screens currently do the same thing, provided my's default layer has not been altered. I don't think there's any value to using that intentionally (other than using a relevant name), but there could be accidental name collisions popping up all of a sudden in cases which used to work fine. With images called main_menu for example. The "sticky" solution could have a hide my statement hide a screen on the screens layer, even if the "my" image has only been showed on master. It's always been documented, I think, that the hide statement does nothing when targeting something not shown, and it would possibly alter that behavior by changing how targeting works.

Otherwise, the feature of having the same tag for images on several layers is useful to manage effects not using shaders. TV overlays for example. I occasionally used that, and it's very handy.

Other than that, my problem lies with an opt-in which changes the rules. If it's an opt-in the compatibility is taken care of, but it makes people opting in follow different rules for show/hide targeting. A few lines of code which would work without the option would yield different behavior with it, and vice versa. That's what would mess up help channels. If we were introducing such a change, we should be calling one of them "the new way" and declare the other obsolete. TLDR regardless of which rules I think are the best, I think we shouldn't maintain two parallel sets of rules for the same statements.

[morbil]

Unfortunately, if you look carefully at your example when the sprites move on top of each other, you can see that an instance (for lack of better terminology) of the shader is actually being applied to them both independently, and relative to their bounds.

Ah, your description was a little off then. It's not shader applying to multiple displayables, it's applying to the rendered texture of multiple displayables. So, single shader, single displayable context.

This is similar then to my earlier request to provide a displayable interface for layers, or a generalized render target.

I think the "sticky_tags" idea could work, however, I think some concept of a compositor within a layer could solve part of both issues. The compositor would allow you to assign drawing attributes like Model but also allows you to assign tags (which include screens) to it in a transient fashion. Something like:

compositor comp:
    parent layer master
    # Each shader acts as a different pass
    # Not a real example, just wanted to demonstrate syntax
    shader 3pood.color_grade:
        texture night_lut # image ref
    shader 3pood.desaturate
    shader 3pood.chroma_shift:
        u_shift_amount dynamic anim_shift
###

# Normal
e smile "Hi"
r "Hi"
comp.add etag, rtag
# Comped
e angry "What the heck!?"
r "Oof"
comp.remove etag
# e not comped, r comped
e happy "That's better"
r "But I'm still stuck"

It still doesn't solve the need for a separate render target displayable, as there's still no way to reference the results in another draw context, nor a way to maintain the results of the previous frame's draw calls. But it does simplify post-processing and allows additional texture referencing without needing a CDD.

It might not be needed for your example, though. You could define a secondary tag for the items you want to post-process, and define a character referencing that alternate tag, which then just requires typing a different say argument rather than onlayer X every time. It's hard to say how well this works with z-buffer though (due to the interaction of different layers and zdepth), but that would be true even in the "sticky" layer concept.

[renpytom]

I'm sort of thinking about this - took a while to get some time to wrap my head around it.

I think it probably makes sense to have Ren'Py have the ability to have detached layers that are available through a Layer displayable, making the contents of the layer something first class.

We might want a less-clunky interface for accessing layers. I'm wondering if in might be a good keyword for this, such that:

show eileen happy in master
show eileen happy onlayer master

Would be equivalent statements.

If we go in this direction, then I think sticky layers would be useful, perhaps with a list of layer names that should be sticky, defaulting to "master" and the detached layers.

[Gouvernathor]

If I understand correctly, it would yield these new rules :

• New virtual or "detached" layers can be declared, inheriting the layer syntax and the placement of displayables inside them and all - in essence a glorified non-shrinking Fixed with a given name, with access to the layer syntax for adding and moving its children - but not displayed as layers
• A new Layer callable, invoking a render of a detached layer, as a displayable which itself will be part of a layer
• A new keyword for the show statement, telling it (in addition to normal onlayer behavior) to remember that the tag should be found on that layer, until the tag is hidden

If the new keyword is not meant to add that last behavior change (which would enable the "sticky" behavior if and when the new keyword is used), then I don't believe a simple rename would be a good thing, as we would only waste an additional reserved word. Not to mention that a lot of images and possibly layers are probably called "in" in existing games. I think in is a bad choice also because it already means two things in python : the __contains__ operator, and the for loop syntax. I would rather have no renaming, or something like on which would have less of this confusion risk and would call back on the "onlayer" word. on is also used in screens and ATL though, maybe ol meaning "onlayer" ?

It might seem obvious but the camera and show layer ... at statements should probably be disabled for detached layers.

Also, we need to think about what to do in case of recursion, when a layer is directly or indirectly part of itself, and how to detect this error.

[mal]

I'd have a mild preference for on. It's more analogous to the existing onlayer, reads a little more naturally within the concept of layers to me, while being just as short as in (though admittedly not currently included in the defined keyword list). That said, in also seems reasonable, but while putting something "inside" a layer makes a lot of scene when thinking for layers as containers, I'm not sure it's quite as intuitive for the casual creator. I dislike ol because it doesn't mean anything - there's no way to infer or guess at it's meaning without prior knowledge.

---

Implementation wise, I don't know how feasible it would be, but if the showing/hiding of a Layer displayable could dynamically add and remove (sticky) detached layers from the scene_list, then it could help considerably with clean up in casual use, and avoid processing, or holding data for, (hidden and/or empty) layers all game long when they're only needed for one or two scenes.

image tv = Layer('tv')

label start:
    scene inside
    show tv:                        # tv layer is now in scene list and available for use
        xysize (400, 300)
    show eileen on tv at left       # eileen is now "stuck" on the detached tv layer
    eileen happy "Hello and welcome to Ren'Py News!"
    show eileen glance_down with ('tv': dissolve}
    player "Eh, I'll catch up with the news later."
    scene channel2 on tv
    show drying_paint on tv
    with {'tv': fade}
    player "There's never anything good on,"
    hide tv with ('tv': crt_off}    # tv layer ceases to exist, implicit renpy.scene(layer='tv')
                                    # releasing resources and implicitly hiding all displayables on it.
    player "Time to go for a run."

    scene outside
    [...]

---

It might seem obvious but the camera and show layer ... at statements should probably be disabled for detached layers.

I'm not sure why this would be obvious. While it's true that a lot of what they can do could likely instead by done by transforms applied to the proposed Layer displayable, I'm not sure the value of extra logic to restrict them as in theory they should continue to work without issue within any transform applied to the enclosing displayable. It's certainly possible to imagine a scenario where it could be advantageous - a TV screen layer warped to fix perspective in scene, but internally using the 3D camera to emulate movement of the TV camera doing the filming.

[Gouvernathor]

(for the last part) in that case you have a double thing in that whatever displayables are shown on the tv layer, in your example, the transforms of camera tv: and the transforms applied to the Layer displayable in the master layer pile up on one another. I don't see a point in having that two-fold system, since you have several ways of doing the same thing without it : having a Layer inside a Layer/Fixed and applying perspective transforms to the inner one and a VHS shader to the outer one, or using a transform with a contains: block and applying it to the Layer, or defining an image with Transform(Layer("tv"), ...) and applying a transform to that image. The first one even lets you use show-with-atl to control the inner transform interactively. In my view camera controls how a layer is treated relative to the screen the same way a normal transform controls how a displayable is treated relative to the layer it's in. So since Layer, as a displayable, gets the second one, it doesn't need the first. And implementation-wise it could add complexity in the form of a supplementary wrapper, i.e the Layer callable would not return a glorified Fixed, it would have to return a Transform wrapping a glorified Fixed.

The general question of what happens to detached layers when Layer() is not invoked (or when one gets hidden) is tough, because we need to cover the possibility of having the same detached layer show twice at the same time. Or we could say no to that, but it was requested in another issue to implement reflections or something, and it would be implied by Layer being a displayable.

[mal]

I don't see a point in having that two-fold system

Interestingly, in a way you've answered your own question with your second paragraph.

The camera/show layer at statements being able to be used on a detached layer being shown more than once with different transforms applied (i.e. two or more TV's with different perspective warps showing the same program) would be worth it since the creator would be able to effectively manage just the TV show without the need to do anything to the TV's besides their initial placement.

it would have to return a Transform wrapping a glorified Fixed.

If you look at the code, that's more-or-less precisely what a layer is constructed as when a scene is being built, and re-using that code path would make a lot of sense.

the possibility of having the same detached layer show twice at the same time

I agree that's probably something we should seek to allow, and you're right about the potential challenges that presents with respect to "dynamic layers", but if in general we like the concept of dynamic layers, then while it would need consideration, it doesn't seem like it would be a technical blocker - in so much as there are ways it could be solved.

[morbil]

+1 for on rather than in as it's closer then to onlayer. Both in (in python) and on (screens, ATL) are keywords already, so no matter what there will be context to unravel, so I'd go with the more natural one.

[renpytom]

We can't use "on", as it would take a common word and make it a keyword.

For example, this would break:

show tv on

"in" is already reserved.

[mal]

Closing as both parts of this feature have now landed in nightlies.