Shaders not working for the focused screen in Hybrid layouts

[jklphe00]

What version of melonDS DS are you using?

1.1.0 and 1.0.3

Does the problem happen in standalone melonDS?

I don't know.

What happened?

When using shaders, only the smaller screens in Hybrid layouts have the shaders applied; the focused bigger screen remains unaffected. This happens whether the small screen mode is set to opposite screen or both screens and it happens whether the top or bottom screens are focused. Other layouts appear to be fine; this issue only occurs with Hybrid layouts.

See attached pics. Here I used the scalefx-aa-fast.slangp preset as my shader. Tested with melonDS DS v1.0.3 and v1.1.0. Render mode used is Software.

What should've happened instead?

All screens should have had the shaders applied.

Which platforms do you see the problem on?

• [X] Windows
• [ ] macOS
• [ ] Linux
• [ ] Android
• [ ] iOS
• [ ] tvOS
• [ ] Other (specify below)

Which versions of the OS?

Windows 10

Which libretro frontend are you using?

RetroArch 1.17.0, Steam version

Terms

• [X] I agree to follow the code of conduct.
• [X] I understand that melonDS DS is not developed or maintained by the authors of melonDS.
• [X] I understand that this form is for reporting bugs, not for requesting help.
• [X] I understand that the author develops this core in his spare time, and that he is under no obligation to respond to this bug report.

[JesseTG]

I notice the rounded corners in the focused screen, although it's more subtle. It seems to me that these shaders are being applied, or is there something I'm not understanding?

[jklphe00]

Sorry. Yes, I do believe there is a misunderstanding here. The intention of the shader scalefx-aa-fast is to smooth out the pixelated look. Here's a screenshot of what the intended look is in the Left/Right layout:

Now, compared to the Hybrid layout:

So, although the focused screen does seem to have some subtle rounded corners, it appears that the shaders are applied incorrectly to just that screen. If I had to guess, maybe the shader is being applied once to the entire layout when it should be applied separately to the focused screen and the smaller screens due to different resolutions/pixel densities of the different screen sizes? I don't truly know, but that is my guess into what is happening since the smaller screens in hybrid mode have the shader appearing as intended.

Edit: Just a quick note. I also tested the xbrz-freescale shader with the same outcome.

[jklphe00]

(Please ignore me closing and reopening this issue. I clicked the wrong thing.)

[JesseTG]

Ah, thank you for the clarification.

[JesseTG]

Now that I have the chance to resume addressing bugs...

Cores don't have direct control over shaders, so I can't apply them differently for each screen. You are correct in observing that RetroArch isn't differentiating between the two screens.

I don't think there's much I can do here, but I'll ask around before I definitely say "no". I think a variant of those shaders that treats the hybrid screen specially may be necessary.

[JesseTG]

Unfortunately, there isn't much I can do about this at the moment. However, the following conversation I had on the Discord server suggests some alternatives:

fishku — Today at 11:47 AM My guess is that the two screens are composed into one image, and that gets sent to the shader, and the result above is what you'd expect. The proper way to handle this is to apply the shaders before composing the two screens. But I don't know about the underlying implementation if that is even supported. In theory, you could work around this in the shader itself, but the shader would need to be aware of the exact screen layout to apply different processing in each part of the image. That's not a good solution.

JesseTG — Today at 11:49 AM That was what I thought, but I wanted to seek a second opinion before I closed the ticket

fishku — Today at 11:51 AM What I would advise them to do is open a ticket on the RA side instead, since the problem lies there. Do you know if there is an API in RA for multiple screens?

JesseTG — Today at 11:51 AM Nope, there is not. RetroArch predates good DS emulation, so no one thought of it as a use case

fishku — Today at 11:52 AM Another idea: It might be possible with a combination of shaders. Set melonDS to render the screens side by side. That way, they have the same resolution in the composed image. Then, apply a shader pass that does the desired effect, like scale FX. Finally, apppend another shader that does the layout correction. I know there are some around, but I don't know which ones from the top of my head. Probably something inside handheld/.

JesseTG — Today at 11:53 AM Which repo holds all the shaders? I think if OP is going to open a ticket, that would be the best place

fishku — Today at 11:53 AM In that case, the shader only has to make very mild assumptions (i.e., screens are either stacked vertically or horizontally), and apply its own layout that the user can configure, albeit with a different interface (shader parameters)

JesseTG — Today at 11:54 AM I have plans to generalize the screen layout functionality; that'll go a long way towards making the core play nice with shaders

fishku — Today at 11:54 AM The "proper" fix discussed above would require refactoring in RA / libretro itself and then in melonDS, and would be independent of the shaders themselves.

JesseTG — Today at 11:54 AM Yeah, that would be a major undertaking.

fishku — Today at 11:55 AM If you want to go with the alternative route I have outlined above, it can be implemented in the slang and/or GLSL shaders: https://github.com/libretro/slang-shaders https://github.com/libretro/glsl-shaders

hunterk — Today at 12:02 PM the ds-hybrid-view shaders take normal top/bottom DS output and do the compositing themselves there's a scalefx version and a sabr version

I will close this ticket. I apologize for the inconvenience.