Video dropouts when using HDMI switcher

[nikitalita]

This HDMI Switcher I am using is having some severe video dropouts when I attempt to use it with this: https://www.amazon.com/gp/product/B07MJ783KG/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1

Every time I perform some some action that results in horizontal scrolling, the video drops out momentarily. If I stay still, I experience no dropouts, but as soon as I move, the video stops displaying for about a second.

I experience no dropouts at all if I hook into the TV/monitor directly. I have no problems at all with any other HDMI devices using this switcher, including a MiSTer.

This HDMI switcher has an undocumented HDCP stripper (like pretty much all HDMI switchers), which might be a contributing factor here.

70 might be related, since this only occurs when horizontally scrolling. This makes me think the underlying issue might be HDMI timing problems.

[therealquaid]

Do you know the vsync_adjust setting you have on the MiSTer? And if any of the less compatible vsync modes on the MiSTer produce any problems with your HDMI switcher?

Would you be willing to try using the SNESTang module on your Tang to see if it produces the same problem? It might help find if it's the frame buffer that is causing this. NESTang uses a one frame buffer. SNESTang uses less than one frame for buffering but it pauses on each frame to try and keep the frame in sync with the HDMI output.

[nikitalita]

Do you know the vsync_adjust setting you have on the MiSTer? And if any of the less compatible vsync modes on the MiSTer produce any problems with your HDMI switcher?

I can give those a shot.

Would you be willing to try using the SNESTang module on your Tang to see if it produces the same problem? It might help find if it's the frame buffer that is causing this.

Yes, I'll give that a shot tomorrow.

NESTang uses a one frame buffer. SNESTang uses less than one frame for buffering but it pauses on each frame to try and keep the frame in sync with the HDMI output.

What do you mean by this? Do you mean that the video output lags behind one frame to ensure synchronization with the HDMI output?

[therealquaid]

For the NESTang there is a one frame buffer which means there will be one frame of extra lag. The scanning speed is different from a NES's output and HDMI at 720p. The one frame buffer is there to try and keep things in sync and compatible with modern screens. But no pausing (should be) needed and isn't used in this case. SNESTang only uses a 16 line buffer due to memory constraints on the Tang Nano 20k. But because of the different scanning speeds it pauses the SNES and waits for the HDMI to catch up.

Further explanation is available here: https://github.com/nand2mario/snestang/blob/main/doc/design.md#snes-video-to-hdmi

[nikitalita]

I see. Is there no way around that? I was hoping to use this for competitive play and speed-runs and those necessitate frame-perfect input.

[therealquaid]

There ultimately are ways to reduce it to a minimum. This is one of my hopes for this project too. I play NES Tetris at high speeds (18,19, and hopefully soon 29) and need frame perfect inputs for certain techniques too.

As long as the monitor has very low input lag, and supports the frame rates of the original NES (slightly above standard NTSC at ~60.098). This is where problems tend to come in. Some monitors don't support it at all and some will accept it but drop a frame every so often.

There are a few ways around this. The RetroUSB AVS (FPGA console) puts out at a standard 60hz. As does a modified NES with Hi-Def NES addon by underclocking the CPU slightly I believe.

The MiSTer has some vsync modes that add 1 to 2 frames of lag, and some that go down to as little as a few scanlines of delay. Which is well under a frame.

If you can get the buffer to be as low as a few scanlines or get the output from the "console" to come out at closer to a standard rate it should be possible to get an extremely low lag environment set up.

The NES has an inbuilt ~16ms delay on each frame. Then the CRT draws the picture from top to bottom in about 16ms. So added together on a CRT you have from 16 to 32ms of input lag. Adding another frame to that definitely makes me miss certain moves in Tetris because I'm used to playing it on a CRT. I think it is possible to get this project down to basically indistinguishable levels of input lag, from the original console on a CRT, as long as the monitor has very low lag.

Sorry if you already know most of this but hopefully it helps.

[therealquaid]

To clarify you can still hit the frame perfect inputs with the Nestang but you would end up used to hitting them one frame later. Because the lag is consistent it is still technically doable in the current state. If not ideal. 1 frame isn't too big of a problem or too much harder but 2 or 3 frames starts to get seemingly impossible for me, at least in Teris.

But ideally I still want as close as possible to the original on a CRT as I can get.

[nikitalita]

There ultimately are ways to reduce it to a minimum. This is one of my hopes for this project too. I play NES Tetris at high speeds (18,19, and hopefully soon 29) and need frame perfect inputs for certain techniques too.

Excellent, this is my primary motivation too.

As long as the monitor has very low input lag, and supports the frame rates of the original NES (slightly above standard NTSC at ~60.098). This is where problems tend to come in. Some monitors don't support it at all and some will accept it but drop a frame every so often.

There's at least a way to test the lag of LCDs deterministically, here: https://github.com/MiSTer-devel/MiSTerLaggy_MiSTer

Testing for capabilities for non-standard framerates would be harder. Do you know of any in particular that do this?

This also makes me think, if we are unable to obviate this requirement, perhaps adding VGA out and using a CRT computer monitor might be easier and less expensive than to find an LCD with all these requirements, if we have enough LUTs to do it. There's a VGA encoder for another NES FPGA implementation here: https://github.com/brandonpelfrey/ice40-nes/blob/master/src/mod_vga_encoder.v

Sorry if you already know most of this but hopefully it helps.

This is super helpful, thank you!

[therealquaid]

I don't want to sidetrack this issue page too much so I continued the conversation over on a discussion page on this topic. https://github.com/nand2mario/nestang/discussions/66

[nikitalita]

Ok, I've tested this with both the MiSTer and the Tang SNES core.

MiSTer configuration:

• going through the switch
• with the NES core
• with vsync_adjust=2 (native framerate)
• VRR=0 This works without a hitch, and the video info shows that both the native output and the TV are at 60.1hz (which means my plasma TV can handle the refresh rate, which is very nice).

Tang SNES core also works without a hitch.

Testing the NES core some more to narrow down the issue:

• It's not just horizontally scrolling, it's seems to be any screen update with any significant amount of changed pixels; it tends not drop out on the pause screen if the pause screen is static, although this behavior is inconsistent:
  • Donkey Kong Jr, which has a non-scrolling screen, will experience drop-outs during gameplay, but will not while pausing
  • Super Mario Bros 2 drops out during gameplay, but will not while pausing (even though it has a flashing "PAUSE" message)
  • Castlevania, despite having a static pause screen, will drop out regardless
  • Super Mario Bros 3 doesn't drop out on the map screen or the start menu, but will drop out during gameplay, even while paused.
• It can be intermittent; if I turn the TV on and the switcher is already set to the TANG HDMI, it works ok, though not without semi-frequent hitches. If I switch away from it, and then I switch back, then I start getting the dropouts.

Again, it doesn't have this problem when I'm connected directly to the TV.

Also, another issue that I found that I thought was unrelated, but now I think might be: another of my monitors will refuse to output sound with the NES core. I had never tested sound with this display before, so I assumed it was just a problem with the monitor. However, when I was testing to see if the SNES core loaded correctly, I noticed the display would happily play sound with that.

[therealquaid]

Thanks for testing :+1: So I think the problem might be somewhere in the nes2hdmi.sv file (the buffer and scaling). The HDMI implementation itself looks to be identical on the NESTang and SNESTang core.

I think the simplest fix for now is to port over the upscaling from SNESTang to use with NESTang. This will give multiple benefits:

• Fix the dropouts on your HDMI switcher
• Fix the audio problem
• Likely fix this issue https://github.com/nand2mario/nestang/issues/70
• Reduce input lag to very low levels (while also changing the output frame rate to 60hz instead of 60.098, which will make it more compatible with TV's and monitors for casual users)

The SNESTang scaler can be found here: https://github.com/nand2mario/snestang/blob/main/src/snes2hdmi.v

I think further down the line porting over some of the scaling methods from the MiSTer would be useful for advanced users. Especially the vsync_adjust=2 with minimum latency. Nand2mario wants to implement switchable options and I think being able to toggle the scaling and sync options would be great.

As for the audio problems the only obvious difference I can see is the output sample rate. 48000 on NESTang 32000 on SNESTang I don't know if this is related or if it's more a problem of the imperfect syncing.

[nikitalita]

That would be quite excellent, good luck <3

Just curious, is there a discord or something where development is being coordinated? EDIT: there is, but the invite link is expired

[therealquaid]

Thanks :smile: Trying to figure this scaler out is taking me down a rabbit hole of clock timings and syncing signals. Let's see how much progress I can make.

I haven't been on the NESTang Discord but I think almost all the work has been done by Nand2mario (very appreciated). With Fjpolo contributing to it. And now you :smile:

[fjpolo]

@nikitalita here's the Discord Invite though not much going on lately

[nikitalita]

The issue seems to have gotten worse with the most recent release; now I'm experiencing dropouts on every display that I try this with, switcher or no. Has anyone else experienced this?

[therealquaid]

That's strange. Is it still not happening without the switcher and on the other displays when using the Snestang module? Or if you roll back to the earlier Nestang? I don't think anything changed with the video or HDMI settings on the newer release of Nestang. I'm starting to wonder if it's a problem with your Tang Nano itself since it's happening on multiple displays.

[fjpolo]

I use this chinese bugger:

and honestly after I started working on the cheats engine, I've been seeing random glitches where the screen turns completely black for 1-1,5[S] and then it comes back. I thought it was the cheats engine introducing a delay in the system and maybe there was a timing issue there, but I've been having it also without the cheats engine.

[therealquaid]

Interesting. We are thinking something in the framebuffer implementation is causing it. Maybe because of the non standard NTSC frame rate? In the Snestang module Nand2mario implemented 60 fps and a smaller 16 line buffer and this one seemed to be working fine for Nikitalita.

Would you be willing to try with the Snestang module to see if there's any difference on yours?

I've been trying to port over the Snestang buffer to the Nestang but I have been having trouble figuring it out properly.

[fjpolo]

I'd be happy to but I don't have a compatible controller for SNES, so can't properly test