search

libretro / beetle-psx-libretro

Standalone port/fork of Mednafen PSX to the Libretro API.
GNU General Public License v2.0
311 stars 131 forks source link

Feature request: allowing users to manually select Region / Video Mode (PAL, NTSC, AUTO) #67

Closed Ryunam closed 3 years ago

Ryunam commented 8 years ago

This is to kindly inquire about the possibility of exposing an option to select the Region / Video Mode in the Core Options of beetle-psx-libretro, thus allowing the user to either force a specific format (PAL / 50hz or NTSC / 60hz) or leave it up to the automatic region detection normally performed by the emulator.

The reason behind this request lies in being able to circumvent the technical limitations that affected the PAL-exclusive localizations of several classics. Back in the day, PAL PS1 games were released on the market in a crippled form that capped them at 50hz, making the overall framerate 17,5% slower if compared to their respective NTSC counterparts. Furthermore, because of the intrinsic difference between the two formats and the different number of lines displayed, PAL versions were often plagued with black borders.

While it appears that the latter effect is mitigated in emulation, the former is still firmly in place. That is because, as far as I understand, Mednafen PSX faithfully reads the regional code of each game and then proceeds to run it with the corresponding Region / Video Mode, but this makes it so that some German, French, Italian and Spanish translations are inevitably hindered by the slower framerate. Being able to choose between modes would allow a wider array of options, while keeping the goal of accuracy.

Thanks in advance!

Ryunam commented 4 years ago

Okay @simias, I tested your build from the dev/pal-60fps branch, but it doesn't seem to show any difference compared to the normal core. All PAL content I have tried plays at the usual PAL speed (50Hz) and with black borders, which is another typical limitation of PAL PS1 games.

Interestingly, on the other hand if you use the PCSX-ReArmed core and set the Region to NTSC in the core options, that core will play any and all PAL content at 60hz and without black borders at the top and bottom. I understand though that the two cores have an entirely different codebase, so I don't know if it's a useful comparison.

The games I've tried are:

hizzlekizzle commented 4 years ago

mednafen has a region-picking (psx.region_default) option, but I'm assuming it doesn't do what we need to do? I've messed around with that option in beetle-psx but haven't gotten it to actually do anything.

simias commented 4 years ago

Okay @simias, I tested your build from the dev/pal-60fps branch, but it doesn't seem to show any difference compared to the normal core.

So that's because I was attempting the homeopathic approach, given that I just noticed that I created the branch but forgot to push the actual commit making the change. So I guess you passed the double blind test and placebo emulation doesn't actually work.

Interestingly, on the other hand if you use the PCSX-ReArmed core and set the Region to NTSC in the core options, that core will play any and all PAL content at 60hz and without black borders at the top and bottom.

Well if somebody can describe precisely what PCSX-ReArmed is doing I might be able to port it into beetle.

From that description it means that the timings shift completely to NTSC and you end up with true PAL60? I was explicitly trying not to do that because it would break with games that output actual PAL without bars (those would get cropped) but that's definitely doable too.

simias commented 4 years ago

I actually pushed the commit now by the way, if you feel like testing again. You should definitely still get black bars but the framerate should go from 50 to 60FPS for PAL games (and the timings for NTSC should be subtly messed up, so don't worry if that doesn't work correctly).

Ryunam commented 4 years ago

So that's because I was attempting the homeopathic approach, given that I just noticed that I created the branch but forgot to push the actual commit making the change. So I guess you passed the double blind test and placebo emulation doesn't actually work.

Haha, that's good, I have a long history of chasing NTSC imports back in the day because they lacked both the slowdown and borders that plagued PAL releases, so I guess I'm the right person to test this. :)

From that description it means that the timings shift completely to NTSC and you end up with true PAL60? I was explicitly trying not to do that because it would break with games that output actual PAL without bars (those would get cropped) but that's definitely doable too.

Yes, what ReArmed is doing seems to be exactly that: upon setting the "Region" to NTSC, it is forcing PAL games to behave as if they were NTSC. The black bars disappear and the framerate is adjusted accordingly.

One way to accommodate all use cases would be to have a core option for Beetle ("Force NTSC timings") with the following values selectable:

Ryunam commented 4 years ago

@simias, I tested your commit cc4fa28f6764acfa84ed3eda0086fab74a4cd445 and I am happy to confirm that it is working as intended. PAL games now play at 60fps.

One thing I noticed is that some games (like Final Fantasy VIII) do not exhibit any desynchronization at all. Both normal gameplay and the FMV cinematics play perfectly fine and never lose sync. Other titles (for example Bishi Bashi Special among the games I've tested) seem to have the issues you experienced with Spyro. The audio is cut off at certain points because the video is now faster and the sound doesn't keep up.

Given that this is most likely caused by the different methods used when the developers converted each title from NTSC to PAL, perhaps we would need ideally another core option that lets the user adjust the SPU speed? I am not sure about this, the only evidence I can bring to the table is that some content does remain in sync, some other content has issues with this (and this probably happens on real hardware too, with the import bootdiscs).

simias commented 4 years ago

Thank you for taking the time to test it.

Adjusting the SPU speed would result in sped up or slowed down audio though, that's not great. Besides in the PSX the SPU and CD-ROM are pretty tightly coupled so messing with the timing of one means messing with the timings of the other which can increase the likelihood of game breakage.

Do these games have the same issues when played in "forced NTSC" mode, for instance with PCSX-R? If not that might be a better way to do it, although I would be rather flummoxed if what was the case.

Ryunam commented 4 years ago

Okay, I have performed some further testing and comparisons between current Beetle PSX with your commit applied on it and the latest version of the PCSX-ReArmed core.

On Beetle PSX with your PAL 60 patch:

On PCSX-ReArmed with Region set to Force NTSC:

On real PS1 hardware, using a bootdisc that can force video mode (such as Import Player Light):

All of this brings me to say that I would personally like a core option like the one I described above: "Force NTSC timings" with 3 options: OFF (default behaviour), Framerate (your commit), All timings (framerate + no black bars). This would most likely satisfy everyone from a video standpoint.

As for the audio desyncs, unless there is a way to completely prevent them from happening and given that some titles work well without desyncs and some others have this issue, I would find it an acceptable compromise for now.

simias commented 4 years ago

Thank you for the extensive testing, that's very useful, at least we know where we stand with this. It's too bad for the audio sync issues but I strongly suspect that there's no perfect solution that doesn't involve game-specific hacks. Running the video faster is bound to create issues for games that rely on specific video timings to keep the audio in sync. The solution would then be to run the audio itself faster but at this point you might just set your emulator to run ~1.2x faster than normal and get a similar result.

Which renderer did you test with beetle by the way? Only software or hardware as well?

Ryunam commented 4 years ago

I have only tested the software renderer on Beetle PSX this time around. I think I remember the OpenGL and Vulkan renderers being a bit more lenient with pre-patched NTSC-forced CD images compared to Beetle PSX software, but I might be mistaken on that.

EDIT: Disregard that, I have just tested the Vulkan renderer and it is behaving exactly like the software renderer.

simias commented 4 years ago

I've updated my branch (you'll need to force update if you have the previous version since I rewrote the previous commit): https://github.com/simias/beetle-psx-libretro/tree/dev/pal-60fps

I added an option to override the PAL video timings with 3 values as you suggested:

Unfortunately while Fast PAL appears to work correctly, Force NTSC doesn't even manage to boot the BIOS. I must have messed something up or maybe PCSX-R's implementation is not as simple as I surmised.

Ryunam commented 4 years ago

Just tested your updated branch and indeed Fast PAL is working perfectly (it's great that it does not require a restart actually), but Force NTSC is not. I also tried setting Skip BIOS to ON in the core options, but nothing, the Force NTSC value does not seem to be functioning right now.

rz5 commented 4 years ago

@Ryunam - Thank you for the real hardware tests. Knowing the benefits and limitations of this hack and how it compares against the same type of hack in other emulators and on a real PS1 will help a lot in bug triaging if and when this gets added.

ggrtk commented 4 years ago

On PCSX-ReArmed with Region set to Force NTSC:

  • PAL games run at 60fps;
  • The picture fills the screen entirely with no black bars;
  • I can confirm that the audio falls out of sync with this core as well on those same titles that showed a desync with Beetle PSX. There is a very slight difference in terms of where the audio is breaking, but I assume this is down to the difference in accuracy between the two cores. In general I can say that the desynchronization does happen here as well.

I'm not sure if PCSX ReARMed is a good point of comparison since from what I've seen, the GPU plugins available on it don't seem to properly emulate the display range control commands which is probably why you're not seeing black bars, even for games that are supposed to have them. PCSX is known for audio desynchronization over time in games like the DDR series as it is less timing accurate, which is likely where the differences in audio are coming from.

On Beetle PSX with your PAL 60 patch:

  • PAL games run at 60fps;
  • The picture has black bars at the top and bottom (this is because, as you previously explained, you avoided altering the timings completely);
  • Audio falls out of sync on certain titles, but works perfectly fine on others based on how they were converted by the developers from NTSC to PAL.

Right now the Beetle renderers use the display control values provided by the game itself, so I think that's why you're still seeing black bars rather than it being a timing alteration issue. A soft way to get rid of the black bars would be to use the Initial/Last Scanline core options. Unfortunately the number of scanlines you need to crop for PAL games are going to depend heavily on how the game was converted.

We could probably extend the Core Aspect Ratio option to let you force an NTSC-esque aspect ratio while running PAL content so with a little tweaking you can get a somewhat NTSC experience with your PAL content.

simias commented 4 years ago

So, if I understood correctly from your tests, our "fast PAL" option and PCSX-R's "force NTSC" option behave fairly similarly in games, the only difference being that "fast PAL" keeps the full PAL resolution (which is bad for games that kept the NTSC resolution with black bars, but good for games that actually bothered to use the additional lines).

If so maybe the simplest solution would be to only have "fast PAL" and then play with the various cropping options to remove the bars when needed?

It would have the advantage of being fairly non-intrusive in the codebase since we already have cropping options (AFAIK, I've never used them) and the fast PAL option only tweaks two lines of codes in the GPU.

Forcing NTSC is a bit messier because it changes many details in the way the GPU functions (different clock divider ratios in particular) which is why in my current non-funcitonal implementation I forced a restart. Savestates can also potentially end up being corrupted if you save in "force_ntsc" mode and load without or vice-versa (it will probably work most of the time, but it's potentially unreliable).

simias commented 4 years ago

@ggdrt arg, I should've waited a couple of minutes before replying because you've answered some of my questions:

I'm not sure if PCSX ReARMed is a good point of comparison since from what I've seen, the GPU plugins available on it don't seem to properly emulate the display range control commands which is probably why you're not seeing black bars, even for games that are supposed to have them.

I should've thought about that, in hindsight it would completely make sense. That means that my force NTSC option in its current implementation is doomed, I'd have to override the display range commands as well.

It may also explain why the games crash: if the display range is greater than the effective resolution the "vsync" IRQ never triggers, that may explain the freeze. If PCSX-R doesn't bother to emulate the video timings accurately it might just hardcode an IRQ at the end of every frame without checking if the blanking has really been reached as configured.

We could probably extend the Core Aspect Ratio option to let you force an NTSC-esque aspect ratio while running PAL content so with a little tweaking you can get a somewhat NTSC experience with your PAL content.

If it's workable I think it might be the least intrusive and most elegant solution. I'll rework my patch to remove the force NTSC option for now, it's not like it's useful anyway.

Ryunam commented 4 years ago

Thank you to both @ggdrt and @simias for the exhaustive answers and for providing more insight about the inner workings of this core and the PS1 hardware itself.

Based on all the information gathered so far, I would agree that the "Fast PAL" option + a new Core Aspect Ratio value for PAL titles with black bars + some title-based tweaking that each user can then apply with the scanline options would basically solve this issue for good. I will be eagerly looking forward to any new development on the matter!

simias commented 4 years ago

I've cleaned up my patch, removed the broken "force_ntsc" option and opened a PR.

I don't know if we want to merge that now or wait for the cropping option as well. Technically they're independent from one another (we might want the crop even at normal PAL framerates after all).

Papermanzero commented 4 years ago

In terms of feedback I would merge it if it is ready. As you said both features are independent, and users can already try the speed up feature.

Ryunam commented 4 years ago

Personally I am definitely in favor of merging it, so that also other people can test it and share their experience. Once we have the implementation of the cropping feature as well, we can then consider this issue officially solved.

simias commented 4 years ago

I think that technically I can merge it but I haven't really contributed a lot to this repo in a while so I'd prefer if a more active contributor could review it at least. In particular I've seen that there's been a big refactor for the core option code with internationalization and all that, I don't know if that needs to be updated before merging.

ggrtk commented 4 years ago

Fast PAL core option is merged, I'll add the PAL to NTSC aspect ratio conversion soon.

These features might be a bit unstable at first as we get things settled.

ggrtk commented 4 years ago

Force NTSC aspect ratio setting under the Core Aspect Ratio option has been added.

I made the line clock counter calculation a bit more accurate using the interlaced rates. The speedup is still a bit off for non-interlaced content though. We might be able to do better by checking the interlace bit and scaling the line clock counter depending on that.

I'm not sure how well this hack works with the way frontends handle audio skew in conjunction with how the core currently reports refresh rate. That could potentially be another source of audio issues.

Ryunam commented 4 years ago

Thank you so much for adding both the Fast PAL and NTSC aspect ratio options. I will be sure to now test both features thoroughly and report back in case I notice anything weird or inconsistent.

simias commented 4 years ago

Great work @ggdrt, thanks!

Ryunam commented 4 years ago

I'm sorry for the really late reply to this! I have been testing the two options and I can confirm the following:

ggrtk commented 4 years ago

The progressive PAL signal is ~288 lines at 4:3 aspect ratio while the progressive NTSC signal is ~240 lines at 4:3 aspect ratio. Many games that were ported from NTSC to PAL keep the same internal framebuffer heights but fill in an extra 48 lines of black in video output for the PAL conversion, meaning that the PAL ports look squished vertically or stretched horizontally.

The Force NTSC aspect ratio just unsquishes/unstretches it back to the aspect ratio that you'd see if it were an NTSC game, but doesn't touch the extra PAL lines. Those have to be manually adjusted by the Initial/Last Scanline core options. We don't automatically remove the bars because many NTSC to PAL ports have significant differences in where the image is positioned -- some games don't even vertically center properly. This option shouldn't be needed for games that actually take advantage of PAL height like Spyro or Crash Bandicoot.

simias commented 4 years ago

It's getting off-topic but mednafen's emulation of proper TV timings seems more trouble than it's worth in my experience. Very few games actually benefit from emulating that properly (as far as I know mainly "screen shake" effects in a few games). It's probably good to have for those games but it makes for a lousy default. We basically have code that bothers to add black lines and then we add options to manually get rid of them. That's a bit silly.

Ryunam commented 4 years ago

Thanks for the explanation, ggrtk. I was under the (wrong) impression that the Force NTSC option was supposed to eliminate the top-bottom black bars, but it makes perfect sense how that can't be an automatic thing and I now understand what purpose is served by the aspect ratio feature.

I will keep this issue open for a little while longer, in order to let others report any observation / finding or lingering issues, but I think that with a combination of:

PAL games can now be enjoyed to their fullest.

Thank you all for the amazing contributions so far and especially for the hard work put towards fulfilling this long-standing request.

ghost commented 3 years ago

I'm glad this option is finally available, however it doesn't work as one would wish for it to work for PAL games: https://www.youtube.com/watch?v=COZMT2TzJ5M

PAL 2 NTSC patched games on other emulators will work and behave exactly like the NTSC part in the video. All this Override option does is speed up the game for no reason. The "slower PAL" thing people always say seems to come from the SNES era, however on more "modern" consoles what happens is that a PAL game leaves 5 frames (25 instead of 30) out, ingame this leads to virtually no difference however in scripted (fps dependent) cutscenes as well as FMVs this will lead to a sort of "teleporting" visual, an example can be seen here: https://www.youtube.com/watch?v=HPuBjkFYoOw&t=171s

If you can't see it that easily, simply lower the videos speed to 0.25s and you will see that every second the video skips a few frames, leading to a cut like / teleport effect caused by it being the PAL version.

A PAL 2 NTSC patched game on an emulator that accepts it will not have any of those issues and will behave just like the original NTSC version.

Simply speeding up the PAL version to 30 FPS does not get rid of the overall issues and will only cause games to either run too fast or cutscenes to break.

trostboot commented 3 years ago

The "slower PAL" thing people always say seems to come from the SNES era, however on more "modern" consoles what happens is that a PAL game leaves 5 frames (25 instead of 30) out, ingame this leads to virtually no difference

Citation needed. There are many, many, many 5th and 6th generation console games that are simply slowed down and squished for the PAL version, because game speed being tied to FPS was commonplace. Proper adaptations are a rarity. I remember Gran Turismo 4 being praised for being a rare example of a decent port, and that's a mid-to-late 6th gen game.

Ryunam commented 3 years ago

Closing this for now, as the feature has been implemented in this core for quite some time and so far has been working reliably, albeit with a few inconsistencies that inevitably derive from the mere fact of forcing a game hard-coded at 50hz to play at 60hz (ie de-synced movies in certain instances, or the skips that were mentioned before). These issues also happen on real hardware whenever using any software / serial-port device that forces a specific video format and they're also highly game-dependent.

I speak from direct experience and a lot of testing when I say that lot of PAL PS1 games were not optimized at all, so having this option is highly beneficial in all such cases and the issues described above do not constitute an example of this feature possibly malfunctioning.

Thank you to everyone involved for adding this override!