FMV Enhancement Pack issues

[Psycho-A]

I don't know, it it still currently supported or it's a no-return thing already (I'm a common user and didn't track the development), but still... Videos from this pack looks good but they have several drawbacks and issues:

1) The brightness range of all videos is incorrect for PC. The original stuff from PS2 is recorded in 16-235 levels range (15-235, to be a bore) which is normal for analog TVs output or probably were corrected by the PS2 hardware itself during play, - but looks totally pale on the PC with full-range monitor and full-range game environment. White level looks dim, black one is not black, but dark-grey. Konami considered that due to porting game to PC, so although bink FMVs looks generally bad, they mostly have a correct brightness levels. In HD Collection it was considered too. It's strongly needs range expansion from 15-235 to 0-255 before encoding. The only videos which doesn't require that, are credits, ending, end_dog, end_wish, flash and water which played in 4:3 aspect and recorded in full brightness range.

2) Bitrate of these videos is groundlessly big without any quality benefits. These well-denoised videos encodes good enough by Bink in 1,5x smaller data rate (or 50:1 compression ratio on current resolution), and there are no any quality penalties noticed by me. Also, 16:9 versions seems to be obtained from already existing 16:10 Bink versions, because all movies have a little greenish tint (which Bink adds a bit on every encoding) and worse quality, but much more larger file sizes on the same resolution.

3) The aspect ratio chosen incorrectly for 16:10 pack versions. If you carefully look on which proportions used when playing on original PS2 (just make a screenshot from PCSX2 and calculate pixel aspect of the image inside of boeders), you'll find that they are exactly 1.5:1 (3:2), but not 16:10 or 15:9 (1.6:1 or 1.66:1). It means that 512x384 FMVs uses stretching about to 576x384 or 512x336 (PCSX2 shows that max width during game play is 512). The PC and XBox versions stretched them to 640x384, so they all look flattened. So, on 2X upscaling process for Enhancement Pack, the actual resolution must be "1152x768" (or 1168x768), not 1280x768 or even 1360x768. Here's the screenshot from PCSX2 running in accurate software mode that shows real picture dimensions:

4) Next issue growing from previous: the Widescreen wrapper works imperfect, and that's why it requires to keep two versions of FMVs for both 16:10 and 16:9 instead of just one in 1152x768 with correct pixel aspect ratio (see above). Also there's no ways to watch enhancement videos on 4:3 monitors without vertical stretching or cropping their sides.

• First of all, it's good to drop FMVWidescreenMode and FMVWidescreenEnhancementPackCompatibility params in favour of automatic calculating FMVs resizing based on selected monitor resolution aspect ratio and letterboxing the video by one of its side. This means that if we choose a screen resolution with a 4:3 ratio, then the videos should be fit by its width, with black frames on top. And if we choose a screen resolution with a 16:9 ratio or other wide one, then videos are fit by it's height, and have black borders on screen sides. This behavior will compatible with original PC FMVs as well as with Enhancement pack ones.

• Second, the extra param like StretchFullscreenFMVs may be added for those who want to see FMVs in 16:9 or 16:10 without black borders in cost of slight stretching. It may also make sense to cap stretching to 16:9 to exclude over-stretching on ultra-wide monitors like 21:9.

• Cropping picture MUST be excluded at all, since these two suggestions above are fully satisfies the possibility to watch all types of videos normally. Or just add some additional param who those who wants to see FMVs in 16:9 with cropping about 9% of image from top and bottom.

[Polymega]

Thank you for your input and feedback with the FMVs. There have been talks of making an "FMV Enhancement Pack v2.0" by its original creator which would include making them at 60 FPS, along with other adjustments/improvements. However, during testings, we've run into an issue where the game doesn't react nicely if you switch it between 30 to 60 FPS mid-game (30 for gameplay, 60 for FMVs).

Until we can solve that issue, any future work for the FMVs has been put on the back burner. We have to make the game play nice switching between framerates for the people who prefer to play the game at 30 FPS (how it was originally played as).

In the meantime, I'll tag @angularGraphics who created the FMV Enhancement Pack so he can read over your technical notes (bitrate, codec, resolution, etc).

In regards to implementation of the FMV Enhancement Pack within the game: This particular mod was made in part by and implemented by ThirteenAG, whose another party over at the Widescreen Fix page found here: https://github.com/ThirteenAG/WidescreenFixesPack/blob/master/source/SilentHill2.WidescreenFix/dllmain.cpp

First of all, it's good to drop FMVWidescreenMode and FMVWidescreenEnhancementPackCompatibility params in favour of automatic calculating FMVs resizing based on selected monitor resolution aspect ratio and letterboxing the video by one of its side.

If/when a "2.0" version of the FMV Enhancement Pack can be made, we can ask if ThirteenAG may be able to dynamically size the videos depending on the game's resolution as detailed in your post. But it is best to wait and ask him this after we can make the game switch between framerates while still playing the FMVs correctly.

Also, 16:9 versions seems to be obtained from already existing 16:10 Bink versions, because all movies have a little greenish tint (which Bink adds a bit on every encoding) and worse quality, but much more larger file sizes on the same resolution.

That is correct. Here are some notes on that.

Second, the extra param like StretchFullscreenFMVs may be added for those who want to see FMVs in 16:9 or 16:10 without black borders in cost of slight stretching. It may also make sense to cap stretching to 16:9 to exclude over-stretching on ultra-wide monitors like 21:9.

I believe this was the plan and would make most sense. Pretty much all other widescreen adjustments have been made with the 16:9 ratio in mind, as that's the most used ratio for consumers.

Cropping picture MUST be excluded at all, since these two suggestions above are fully satisfies the possibility to watch all types of videos normally.

I will take cropping over stretching. Stretching is a last resort for me, personally.

You can read more about some of these talking points here: https://github.com/elishacloud/Silent-Hill-2-Enhancements/issues/13#issuecomment-407188237 and here: https://github.com/elishacloud/Silent-Hill-2-Enhancements/issues/35

[Psycho-A]

@Polymega thanks for answer!

I don't know is 60 fps really needed for a such short cutscenes that're full of ghosting and motion blur effects even for 30 fps, but what I'm sure in, that is the version 2.0 needs all original PS2 videos in correct Pixel Aspect Ratio and no cropping and black pillars, as a source material for all subsequent actions. And since 2x scaling is the best way to exclude any blurs or pixelatings, the size of 1152x768 will best for it (who needs in all these HD standards of 720p or 1280 lines if game stretches it to which is needed anyway?). Having correct PAR movies and automatic resolution-specific letterboxing as a default Widescreen patch behavior would be enough for most users; and besides it's an easiest way to add some additional features after it's done, like user-desired stretching and cropping - than initially adding some workarounds to manipulate by already squashed 16:9 videos.

Maybe, 60fps versions would also require additional pack since the size of sucn FMVs will be even more than current ones, so there will be enough "light" version for most players...

Btw, I did my own FMV pack from original PS2 and Konami's DVD for trailers, using VirtualDub and plugins, and got almost the same results like currently in Enhancement pack 1.0. But it uses original game FMV's resolution with minimum quality losses, which is compatible with Widescreen patch if FMVWidescreenMode=1 is set. Here is it if you interested. Just wanted to say that if will no more appropriate video sources from angular for v2.0, I could try to help a little on it,

[rokweom]

The proper way to handle this is encode the videos without any black bars whatsoever in their correct aspect ratio, and then add letterboxing upon playback. No stretching, no cropping, no information lost. That's the only proper way to play videos with different aspect ratio than that of the display, no matter if we're talking about movies or video game FMVs. Of course there are always these people who complain that a Blu-ray they bought is in 2.39:1 letterboxed, instead of filling their 16:9 screen, but for them, all hope is already lost.

Whether it can be done - probably not. Not easily, at least. Maybe sharing few versions of the patch for different aspect ratios would be a way to work around this.

[Polymega]

The original FMV Enhancement pack was done long ago, possibly predating the SH2 Widescreen Fix project. angular_graphics, the creator of the FMV pack, has expressed interest in making a "2.0" of their work, taking things like rectifying the black bars into consideration, among even more improvements. However, they're holding off on any work until we can find a way to make the game switch between 30-to-60 FPS without issue. Currently, if you play the game at 30 FPS but have the videos play at 60 FPS it causes playback issues.

I can't blame angular_graphics for not wanting to make an updated version of the FMVs at this time because, without being able to bring the videos back to their original 60 FPS, there's not much incentive to do so for the moment. angular_graphics would be the authority on this, but making multiple versions of the FMVs for different aspect ratios might be burdensome, unless you kept it to something like two different versions; 16:9 & 4:3.

[Badore90]

I tried this pack. Excellent quality! Btw I found a problem in the FMV where you meet Maria for the first time (deai.bik/dean.bik): there's a ugly effect on the edges.

[mirh]

Bitrate of these videos is groundlessly big without any quality benefits.

That's bink for you. On the other hand, it has come to my attention from the guy that reversed engineered the format in the first place, that it's actually pretty easy to use something else in its place.

Why hack? Even if RAD discourages it, you can find the API for [directly showing your own decoded frames instead of bink's] (in some open-sourced game engines for example) and write your own wrapper that would accept the calls but do something different instead (like calling an external decoder).

https://github.com/GTAmodding/VCSPC/blob/master/VCS%20PC/VideoPlayer.cpp https://github.com/Nommy228/Might-and-Magic-Trilogy/blob/master/Bink_Smacker.cpp https://github.com/jsj2008/lithtech/blob/master/runtime/kernel/src/sys/win/binkvideomgrimpl.cpp

Yes, like these. You know the prototypes for Bink functions so it should be possible to create a shim DLL that would (for example) load the real DLL, forward all calls to it but on BinkOpen it would also check filename, open a decoder for companion hi-res video file and report a new improved video dimensions. And on BinkCopyToBuffer call it would return a frame for a new video instead of an old one too. This way the sound from old video will be played with new video track. Of course somebody can write a complete replacement that would simply use the same API but do something completely different inside but you need somebody experienced with audio playback programming and DirectX to pull that off.

I also found out people already kind of taking advantage of those apis, even though nobody has yet hooked up ffmpeg or something alike.

[Ales724]

Any updates on this? I deinterlaced & upscaled a PS2 FMV just so i can get a rough idea of how 60FPS FMVs would look in game, here are the results for anyone wondering

video.webm