Video player stretches videos (cutscenes/memories)

[InterClaw]

Would it be possible to temporarily set aspect ratio to 16:9 during cutscenes? And then switch it back to whatever it was when the cutscene ends (e.g. stretched). I guess the same detection that is used in DynamicFPS/DynamicFPS++ can be used for this.

[fayaz12g]

Theoretically yes, I will look into doing this once I have added the other features I’ve promised

[xNicklaj]

If it can help you in any way I'm quite sure that the original theboy 21:9 mod had this feature, maybe you can give it a look and adapt it for any resolution

[fayaz12g]

@xNicklaj I'll take a look at it, do you have a copy of it anywhere so I can test it? Also I appreciate all the UI issues you've found, would you be willing to look into helping me make fixes? I'm also a student learning all this haha

[xNicklaj]

I would honestly not dislike it, the problem is that I have no idea whatsoever on how to start or where to start. I don't despise the idea of learning from scratch though!

Hit me up on Discord (user: nicklaj) and we can talk it through!

[Fruithapje21]

Let me clear up some things. Setting the aspect ratio is done by letting Yuzu scale the final image to fit your screen. The game itself will always output frames with a 16:9 aspect ratio. You cannot 'temporarily set the aspect to 16:9' because you didn't change this aspect in the first place. There is currently no way to change the aspect ratio setting of yuzu on the fly. The only thing you can do is setting the aspect ratio to 'stretch to window' and rescaling the window yourself when the cutscene is playing.

I also had a look at theboy's 21:9 mod. It sets the playback size of the prerendered video to 1280x900 (normally 1280x720). What this does is stretch the video vertically. However, you can still only see the top 720 pixels, so you are effectively watching a zoomed in version of the video. If you were to extend this approach to larger aspect ratios, you will zoom in more and more and see less and less of the original video.

If you, instead of stretching he video vertically, squish the video horizontally, you do end up with the correct aspect ratio and will be able see the full video, but the remaining pixels will be read from out of bounds memory. For 21:9 I ended up with this masterpiece

[InterClaw]

I guess this means the only known way of getting effective 16:9 video in a 21:9 viewport is to reencode the video with pillarboxing then?

[Fruithapje21]

Yes, I'm afraid so. Also good luck trying to distribute those without getting unwanted attention from Nintendo.

[xNicklaj]

You could use an approach where you don't distribute the encoded version, but rather a script that given the original files re-encodes them

[fayaz12g]

My thoughts in the past month on this has been two options, 1) include a part of the script that surrounds the 16:9 video in black bars (letterbox or pillar box depending on vertical or horizontal script), then inversely squishes it back into a 16:9 video and puts those video files into the romfs so that yuzu stretches it back to the desired ratio

• pros: any entered aspect ratio would work without any zooming
• cons: either the program would need to download the original video files (they’d be large, like when the original 60fps cutscene mods came out, and Nintendo would take the source down) or the user would need the romfs dump, which would take a while longer and use up space

2) find the video player the game uses and scale the root pane like the blyt files

• pros: small, fast, easy
• cons: the video player may not be customizable like that, afaik we don’t know what files are responsible for playing the videos and it likely won’t have a root pane that we can scale, making this option impossible.

So I’ve been searching for option 2, but if someone knows better let me know if my search is fruitless

Edit: as I read more it seems option 2 is definitely not possible, but squishing the video could lead somewhere. Would it be possible to set the out of bound pixel range to display black and then horizontally squish an even amount from both sides? Or can you not assign values to it since it’s out of bounds

[Fruithapje21]

I get your idea but I don't think that is how it works. There is only so much you can do by editing the game executable directly. What you are looking for is not some layout file but a shader. I went digging into BOTW's graphics pack for Cemu and found this cutscene fix. I would assume that TOTK uses something similar but I'm not sure where to start looking and whether there is even support for editing these shaders.