Feature Request: Decouple Rendering Resolution & Window Resolution; Implement Spout

[cappelnord]

For a special project sending a large canvas to projection mapping Software is needed. As #49 is potentially too much work for now a quick solution would be to:

• Introduce a SubViewport with a resolution that can differ from the main window resolution.
• Make all Actors children of this SubViewport
• Render this SubViewport to the window (to preview)
• Introduce configuration options to set the dimensions of the SubViewport

In a default case the SubViewport can be the same dimension as the window. This would introduce some render overhead but it would probably be negligible.

Afterwards:

• Implement Spout sending using spout-gd, e.g.: /spout/send $spout_name (attached is an example project for that including pre-compiled Windows DLLs) Godot-Spout-Test.zip

[totalgee]

One issue about having different sub-viewport (or huge, for print) resolutions is the fact that Animatron position/movement commands are currently in pixel units. Same with animation drawing; they are not scaled to be (e.g.) a fraction of the view, so at scale 1 they are 1:1 pixels in the view. On a huge viewport they would look tiny.

This would also need to change, so we can render at arbitrary resolutions. Unless you have other proposals, such as knowing the "real" viewport resolution, and scaling things (display of actors and their movement) based on the ratio between the "sub-viewport" and the actual one. Though this may be a bit hacky.

Also, would sub-viewports be restricted to be the same aspect ratio as the main view? If not, would you want stretching, or cropping/letterboxing to resolve the difference?

[cappelnord]

I imagine it like this:

• Default case (and will be for most users) is that SubViewport and Window is 1:1. If The render size is not expliclty set then also all window resize operations can always affect the SubViewport so the behaviour will overall not change, pixels are sampled 1:1, no artefacts, ...
• Setting the render size to another resolution is an explicit choice (also explicitly choosing the resolution), so users are aware of the change of the coordinate system and it is their responsibility to think according to it (in pixel units of the SubViewport, not the Window). Introducing a possibility to switch into normalized coordinate system could help here but is also not needed.
• Things might look tiny but this is the nature of it (of course an option to zoom in/pan around would be nice but maybe not needed, or something that could be done with a more general solution like #49 could be done programmatically)
• I would letter/pillarbox the display of the SubViewport to the window by default.

[totalgee]

I need to see with @loopier how the viewport is currently drawn as part of the HSplitContainer...I don't quite understand where it's doing the "letterboxing" already. This code editor should instead interact with the new MirrorDisplay node, so it always fills the actual window...or some other behaviour, as desired. At least the basics of drawing to the SubViewport and showing it (scaled) are implemented.

[totalgee]

After discussing with @loopier and looking more, I see it's actually the Project Settings (for the main Window) that are controlling the behaviour that was confusing me, not the HSplitContainer! I was able to figure it out... ;-)

Default case (and will be for most users) is that SubViewport and Window is 1:1. If The render size is not expliclty set then also all window resize operations can always affect the SubViewport so the behaviour will overall not change, pixels are sampled 1:1, no artefacts, ...

@cappelnord I have made a new commit 1f8671c that gives the basic behaviour you describe. If the main content is resized, the SubViewport stays 1:1 with it, (for now) there is no way to make it different. As you resize the main window, the behaviour will be as before, if the aspect ratio is different from what was specified, it will do letterboxing.

Anyhow, if you want to try some tests now with Spout, you at least have a separate texture you can access.

• [x] Introduce a SubViewport with a resolution that can differ from the main window resolution. (it can have a different resolution, but right now it is set up with signals to automatically sync (resize) whenever resizing the main content area)
• [x] Make all Actors children of this SubViewport
• [x] Render this SubViewport to the window (to preview)
• [x] Introduce configuration options to set the dimensions of the SubViewport (added later, with e4e2b11)

[totalgee]

I can try doing the Spout stuff at some point, but not in the next few days...

[cappelnord]

Great, I will check it out!

If the Spout stuff is for now out-of-scope from here on we can also help ourselves with some quick add-on/copy-in to Animatron to make it work for our project. An issue I see with the Spout stuff is that for the project to stay portable to all platforms one must likely compile empty stubs for MacOS and Linux to link against (in addition to check at runtime if it is running on Windows). This could also be solved with platform-dependent conditional compilation à la Unity but this does not seem to be available (yet) for Godot and I also did not yet come up with another solution how to solve this in an elegant way.

Maybe you I also did not know what to look for :)

[totalgee]

Previously I assume (as you stated @cappelnord) that the window resolution would always match the SubViewport resolution, but @loopier wanted to test different sizes. I've made it work (locally) so that when you call /view/size 480 270 (for example) it will make a quarter-sized view, and will scale/stretch it to fit the main window. Likewise when you do /view/size 5760 1080 it will make a 3x-wide HD viewport to render to, and will "letterbox" it. But currently, you don't see black borders (top/bottom), because the "content scale size" for the main window currently remains FullHD sized. Basically, I'm not sure what you'd like me to do with the display main window.

There is a thing called the "content scale size" -- this is kind of like the viewport for the main Window. It's set up to do the letterboxing and so on automatically as you resize the window (the content scale size stays at 1920x1080 always, the way we have it set up now), and this does not need to match the SubViewport resolution (which you will eventually "export" using Spout at its native size). But the UI stuff (code editor and post window) currently seems to be set up to work with FullHD resolution only. I tried changing the "content scale size" to match the SubViewport, and it works, but...if I make the SubViewport a small resolution (like 320x240) then the code is huge and you can't see all the editor on the screen. Likewise, if you set it to a very large resolution (5760x1080) then the code is tiny and impossible to read.

I could leave the "content scale size" at 1920x1080 all the time, so the code editor looks normal, and then the actual SubViewport get stretched to fit into that space. Is that okay? Basically, my question is: what behaviour do you want to see in the main Animatron window when you have SubViewport size to something that is very different (in scale and/or aspect ratio) from FullHD?

If necessary we can have a meeting where I can show what I'm talking about.

[totalgee]

I think I need to change the main window so its content scales with the actual window resizing (rather than being fixed at 1920x1080 or matching the SubViewport size), then the code editor will works fine. Then I'll make the SubViewport (and its mirror node) compensate so they're always fitting in the full window. I think that would probably make the most sense... Any other opinions welcome.

[totalgee]

Here are the examples I was using while testing this scaling feature:

/load default

/create bg default
/color bg 0 0 1
/scale bg 500

/create a default

# half size
/view/size 960 540
/pos a 480 270

# quarter size
/view/size 480 270
/pos a 240 135

# try resizing the main window, see what happens (H/V letterboxing)

# back to 1:1 pixels, sizing with main window content area
/view/size 0 0
/pos a 960 540

# try resizing the main window, see what happens (actor workspace size matches main window)

# different aspect ratio (3x HD screens)
/view/size 5760 1080
/pos a 0 540 # left edge
/pos a 1920 540 # 1/3 way along
/pos a 3840 540 # 2/3 way along
/pos a 5760 540 # right edge

# square aspect ratio
/view/size 1000 1000
/pos a 500 500

[totalgee]

I added another commit e362f8b so actors are created centered correctly using the SubViewport stuff...i.e. regardless of the size/shape of it.

[totalgee]

Summary of what changed:

• You can now set the actor view/workspace size using /view/size sx sy
• You can make the Animatron window whatever size and aspect ratio you like. If you've explicitly set the view/workspace size then it will be stretched to fill the window (keeping aspect ratio).
• If you never set the view/workspace size, or if you reset it with /view/size 0 0, then the view/workspace will match the Animatron window, 1:1 pixels with it. As you resize the window, the view/workspace will resize to match.
• In spite of all this, the code editor and post window will display nicely, at a reasonable size.

[totalgee]

BTW, I got Spout sending working locally. What I'm not sure about is, if I commit this, will it break the Linux or Mac builds? I am avoiding calling into any Spout code by checking the OS at runtime, but I'm not sure if I'll get a compile error on other platforms, since the class Spout won't be defined. I don't know how to do "conditional compilation" in Godot.

One spout-gd issue is that you can't seem to send the Texture directly (such that data stays only on GPU); you need to create an Image (loads to CPU each frame == bad!). Even so, I was able to get more or less 60 fps on my machine with a 5760x1080 texture being sent. It's right on the borderline, and really depends on the data format (for example, it's slower with FORMAT_RGB compared to FORMAT_RGBA, so I changed the SubViewport node to have an alpha background and therefore be RGBA).

I opened an issue on spout-gd, asking how to use its send_texture() functionality...hopefully I'm just missing something obvious.

[totalgee]

@cappelnord I saw how to do it from your example project, with the trick to get the native handle from the RenderingServer, thanks... The trick was I also needed to switch the renderer to gl_compatibility (rather than Vulkan "Forward+"). Now it's nice and fast, even sending 60 fps to the Spout Receiver with a 16k x 12k viewport (or higher)...

[totalgee]

Spout stuff (first version) is checked into main branch. @loopier needs to test on his Linux version to see if it fails to run, in spite of the runtime OS checks I added there. To use it (on Windows), just do:

/spout/send Animatron  # (or any other name you prefer ;-)

[cappelnord]

Thanks @totalgee - tested our use case and this works all fine! On my Mac the project did not compile anymore but changing these two lines in SubViewport.gd should fix. Tested this also on Windows and Spout itself does still work.

# old: spout = Spout.new()
spout = ClassDB.instantiate(&"Spout")

[...]

# old: (spout as Spout).send_texture(handle, 0x0DE1, size.x, size.y, false, 0)
spout.send_texture(handle, 0x0DE1, size.x, size.y, false, 0)

There are still error messages when running the project on Mac as the extension cannot be loaded (which is not pretty). Not sure if there is a quick workaround for that but I can also compile some stub extension at a later point which will silent it (effectively the extension will never run as the OS is checked at runtime).

[totalgee]

I've added the "last" commit for this mini-project. It should hopefully now run on Mac/Linux (thanks for your tip), and I also added a "stop" command for Spout usage. If there's nothing else needed right now, then I'll probably close this issue. Glad it's working for you!

[cappelnord]

From my side for now all good - thanks again! I'll just do a pull request in case I'll compile new binaries (and potentially the stubs).

[loopier]

I just pulled and tested the main branch. I do get the missing Spout addon error, and failing opening it, but it does compile.

But the scene I had scripted in an ocl file gives different results, probably due to the scaling and positioning management. I'll look into it to see how it work and see how I work around it. Thanks @totalgee and @cappelnord . This is an awesome improvement.

[loopier]

Please, forget what I just said. I'm using a tiling window manager. If I disable tiling, the old ocl script renders exactly as before. My bad.