elishacloud / Silent-Hill-2-Enhancements

A project designed to enhance Silent Hill 2 (SH2) graphics and audio for the PC. It also includes scripts to build or modify SH2 audio files (SFX, BGM and Dialog).
http://www.enhanced.townofsilenthill.com/SH2/
zlib License
596 stars 42 forks source link

Download 60fps and 3D SBS stereo videos here! (OLD TITLE:About REAL 60fps FMV videos) #724

Closed NoUserNameForYou closed 1 year ago

NoUserNameForYou commented 1 year ago

EDIT: I have done the interpolation with RIFE 4.6. Download them if you like.

Three files with multiple mirrors

https://www.mirrored.to/files/1IPSMJ9H/movie1.7z_links https://www.mirrored.to/files/0OANA89K/movie2.7z_links https://www.mirrored.to/files/M1WPJS2H/movie3.7z_links

EDIT2: I converted 2D 60FPS story FMVs into 3D Half SBS HSBS with MiDaS and 3DCombine.

Two files with multiple mirrors

https://www.mirrored.to/files/6HDVHNK9/movie3d1.7z_links https://www.mirrored.to/files/BUVYMBE6/movie3d2.7z_links

Assign a key to turn off Reshade's 3Dsuperdepth effect or your 2Dto3D converter software during story FMVs and let the TV/headset handle the 3D.

Polymega commented 1 year ago

Hi there,

Thanks for your message and interest here. 60 fps is on my to-do list but hasn't been something that I've gotten around to yet. You can see a proof of concept I made for it here.

I've cataloged most of my process working on these videos here.

If you'd like to play around with the videos then by all means! Just please note that I am very particular with how the FMVs would be handled and uprezed. There have been a few FMV concepts/edits submitted by other fans in the past that I politely and respectfully passed on. I am happy to pass on the torch of this work to someone else, it's just I'm going to be a real stickler about it. So I want to give you a heads up on that!

I'm assuming you don't use bink smacker software and call it a day?

Currently, the final step in the tool chain is converting the videos to Bink format using the RAD Video tool. This is mentioned in that document I linked above.

I'll be providing you AV1 MP4 files and I'll be using these ffmpeg settings:

It's best to keep the entire process lossless, up until the very end. Every time a video is rendered or converted to a lossy codec, you lose a little more pixel information each time. So if the videos currently need to stay in Bink format, it should be converted to this codec at the very end from a lossless format.

The RAD Video tool for converting formats to Bink is free to download. You can grab it here.

Perhaps they'd be useful when you switch video player engine in the future?

It's not a 100% guarantee at this time that we'll be switching the video player the game uses from Bink to MP4, but I'm hopeful it will be at some point.

And since the vids don't have any grain anymore, would SH2 add its own noise/grain filtering or should we keep the grain intact in the videos?

SH2's FMVs have baked-in noise that would need to be removed. This is needed for a lot of reasons:

  1. If a player disables the noise filter but noise is present in the video, then the noise filter isn't truly disabled like the player wanted.
  2. The baked-in noise in the videos would naturally scale with the rest of the video to fill the display area. This would make the size of the videos' noise grain different than the noise produced elsewhere in the game, making the noise inconsistent.
  3. The game renders noise on top of the FMVs. So you'd have the baked-in noise of the FMVs competing under the noise generated by the game.

And by removing the videos' baked-in noise, it allows the noise generated by the game engine to sit on top without competition. So the noise size/speed/pattern remains consistent throughout.

NoUserNameForYou commented 1 year ago

Currently, the final step in the tool chain is converting the videos to Bink format using the RAD Video tool. This is mentioned in that document I linked above.

Yeah I see the .bik extension but I thought there'd header hex editing etc. under the hood programmer stuff to make the game accept higher quality videos. It seems like it's just replace and play.

It's best to keep the entire process lossless, up until the very end

I work with 8bit PNGs mostly, so that's okay. What I meant was, I'd be providing you AV1 MP4 files (as I thought there'd be under the hood programmer stuff for editing bik files) if you were interested in frame interpolated outcomes, since 900p lossless videos would take like 50GBs (I assume) and I don't have the bandwidth to upload such size.

But then again, since it was just bink'ing them, I could be provide the final files as well.

it allows the noise generated by the game engine to sit on top without competition

Please disregard my remarks about noise/grain. Turns out I had old files (probably still created by you) I updated the pack now, and video files are higher res and denoised.

I'll just leave this project in your hands as we may not agree on how things should be done and create my own 60p files. Thank you for your efforts.

Polymega commented 1 year ago

I didn't mean to discourage you, and I apologize if I did. I just felt it was good to give you that preface.

To give you some ideas: In the past, there were submissions where the videos were worked on but, when it came time to enlarge them, barebones bicubic rescaling was done. The videos should be scaled to a higher resolution, but it very likely needs to be done with the help of awesome filter combinations and/or the aid of AI to help mitigate pixelation when enlarging.

Increasing their native resolutions helps give them a better chance to look nicer in the game when played in HD resolutions. A simple bicubic resize of the videos (whether done by the video editor for rendering or done by the game on playback) runs the risk of having them appear pixelated/"blocky" while also being "blurry" (from the bicubic smoothing of the pixels edges that were blown up). This effect becomes worse the higher the resolution a user plays the game on, so if the videos are still at 640x410 then RIP.

Another submission ran them through a different type of AI upscaling and, when viewing a single, static frame of any given video, it looked really spectacular. But in motion it wasn't very good as character eyeballs and teeth would subtlety "shift" and "wobble" around while in motion. It was very off-putting!

I would absolutely be interested in seeing what you can do for the videos. All of the stuff I mentioned above wasn't meant to deter you, but rather challenge you to see if you can work some new magic into these video. I won't lie: It's a bit of a challenge. The best sources we have for the Story FMVs are compressed M2V videos from a PS2 disc and they're 512x384 interlaced. There is not a lot of data/pixel information to start/work from... It's a challenge!

If you're up for the challenge and want to do some initial tests, I'd recommend experimenting with the gero and rouya videos first, as they're some of the "rougher" ones, in terms of compression and embedded noise.

NoUserNameForYou commented 1 year ago

I didn't mean to discourage you, and I apologize if I did

No worries. You didn't.

I checked your Excel file and only differences I'd make would be small sprinkle of dehalo'ing with VEAI 3 Proteus (it generally helps with the teeth and eyeball shifting you mentioned) and zero AA/deBlur. I'd also use QTGMC with repair for deinterlacing from the beginning as I don't like Topaz's deinterlacer compared to QTGMC.

But in motion it wasn't very good

Since there are not many videos, one could do scene by scene parameters and export as 16bit TIFFs, then merge them because seems like one-solution-for-all parameters provide different results.

Other than that, your files are good. But we desperately need MP4 playback. Assuming we can play 10bit HEVC (files created with "--vpp-deband range=24,thre=6,thre_y=6,thre_cb=6,thre_cr=6,dither=24,dither_y=24,dither_c=24 command for NVEnc should have smooth gradients). Or any other deband would work as well.

The AI upscaler cleaned the files from the noise but introduced banding.

I somehow remember downloading 60p files from NeoGAF's SH2 enhancement guide back in the day. Maybe I remember wrong though. Were they yours?

Polymega commented 1 year ago

But we desperately need MP4 playback.

110% agree. cough @Gemini-Loboto3 😊

The AI upscaler cleaned the files from the noise but introduced banding.

Yeah. There's a very good chance that the Bink conversion itself introduced a good chunk of that color banding. Gemini made a prototype build for MP4 playback and I was shocked at the quality difference between MP4 and Bink... and I liberally compressed the MP4 videos too with the goal of significantly reducing the FMV Pack's overall size. Check this out:

Bink format, 1920x1200, ~30 MB filesize

image

MP4 format, 1920x1200, ~17 MB filesize

image

Pay special attention to the background behind Eddie. That's an impressive difference, especially considering the MP4 version is almost 50% smaller in filesize!

I somehow remember downloading 60p files from NeoGAF's SH2 enhancement guide back in the day. Maybe I remember wrong though. Were they yours?

Um, I don't think 60 FPS versions ever existed. The NeoGAF thread you're thinking of was angularGraphic's FMV pack and that was kept at 30 FPS (well, 29.97 but you know what I mean 😝 ).

Polymega commented 1 year ago

Oh, ignore the letterboxing on that second image; this was all test files at the time.

NoUserNameForYou commented 1 year ago

I'm currently uploading two Bink v1 WIP files for you.

Got the .PSS files from JP DC PS2 game. QTGMC Deinterlaced and debanded them with -filter_complex "deband=1thr=0.05:2thr=0.05:3thr=0.05:b=0:r=-25:c=1" command with FFMpeg. Used Topaz Upscale Proteus v3 with really subtle parameters (things start looking plasticy). Then converted from TV 16-235 to PC 0-255 (crushing blacks, image is darker) then BIK1'd them. Didn't touch the aspect ratio either. Will take a look at that later.

I didn't use RIFE frame interpolation because QTGMC results at 59.94FPS were just fine.

I also tried bik2 encoder and got the files to play with the vanilla game + hacked binkw32.dll. Tried to use bik2 files with EE disabling most enhancements but it's a lost cause, I got it to work with EE too but the game pretty much looked like the Vanilla game. I'm guessing some functions in EE attach themselves to binkw32.dll or at least communicate with it.

I liberally compressed the MP4 videos too

If you're the author of the .doc file you linked, you robably used Hardware Encoding, which is pretty inefficient. I used Aomenc AV1 with speed preset at 6 for the 26 second file, and it ended up as 2MBs at same quality.

Since MP4 is a container and many people use AVC-MP4 interchangibly, I hope Gemini_Loboto introduces his custom FFMpeg with VVC/AV1 and xHE-AAC/Opus capability. It could be optional for people with fast CPUs but low bandwidth. The current 1.6GB pack could very well be 160MBs.

EDIT: I found the open.bik file in Silent Hill 3 (Bonus-Lost Memories 'The Art & Music of Silent Hill'). It's a 720x480 25i video with blackbars (effective resolution of 720x400). If you have the other higher Q vids, please share.

Polymega commented 1 year ago

Didn't touch the aspect ratio either. Will take a look at that later. [...] Here's the .7z file. https://multiup.org/download/e386cac54c416c9feb5652b2e7f57920/movie.7z

Sweet, and thanks! I'm having trouble with the site the package is hosted on. I couldn't find a way to download it without a user registration and, when registering, I can't seem to get a confirmation email from them. It's been about an hour so far. I'll wait some more to see if it comes through.

By the way, where can I find the Art of Silent Hill DVD or at least the ripped, untouched files you mentioned? [...] EDIT: I found the open.bik file in Silent Hill 3 (Bonus-Lost Memories 'The Art & Music of Silent Hill'). It's a 720x480 25i video with blackbars (effective resolution of 720x400). If you have the other higher Q vids, please share.

Yes, give me some time to get that for you. I'm going to try and send that over today but if, for whatever reason, I don't get back to you with this by the end of next weekend, remind me again here.

NoUserNameForYou commented 1 year ago

Sweet, and thanks! I'm having trouble with the site the package is hosted on. I couldn't find a way to download it without a user registration and, when registering, I can't seem to get a confirmation email from them. It's been about an hour so far. I'll wait some more to see if it comes through.

Sorry, it's a POS site that hides downloads behind bunch of misleading ads.

Yes, give me some time to get that for you.

Take your time and if you can't find it, don't. I'm currently creating files for MYSELF but I'll share them here regardless. Since I create them for myself, I'm not striving for perfection and they are not upto EE team's standards.

I'm saying this because I used the opening scene from the open.bik from the 720x400 25i source from the SH3 bonus DVD (not a rip, straight ISO) and it has color difference between it and the rouya.bik. I tried to stitch rouya from different sources, made one alas, it's iffy. I'm still sharing it in half an hour.

Don't bother with the DVD upload. It's not worth it for me.

MarioTainaka commented 1 year ago

But we desperately need MP4 playback.

110% agree. cough @Gemini-Loboto3 😊

The AI upscaler cleaned the files from the noise but introduced banding.

Yeah. There's a very good chance that the Bink conversion itself introduced a good chunk of that color banding. Gemini made a prototype build for MP4 playback and I was shocked at the quality difference between MP4 and Bink... and I liberally compressed the MP4 videos too with the goal of significantly reducing the FMV Pack's overall size. Check this out:

Bink format, 1920x1200, ~30 MB filesize

image

MP4 format, 1920x1200, ~17 MB filesize

image

Pay special attention to the background behind Eddie. That's an impressive difference, especially considering the MP4 version is almost 50% smaller in filesize!

I somehow remember downloading 60p files from NeoGAF's SH2 enhancement guide back in the day. Maybe I remember wrong though. Were they yours?

Um, I don't think 60 FPS versions ever existed. The NeoGAF thread you're thinking of was angularGraphic's FMV pack and that was kept at 30 FPS (well, 29.97 but you know what I mean 😝 ).

I have a bit of an idea for if and when MP4 videos become a thing. I don't know how feasible this is but hear me out. For language packs it would be neat if we could only ship the audio that's muxed into the mp4. This would bring the size down a ton because you wouldn't need to ship muxed videos for every language.

Polymega commented 1 year ago

I tried to stitch rouya from different sources, made one alas, it's iffy. I'm still sharing it in half an hour.

I did the same thing with rouya, lol. I used as much footage as I could from the AoSH DVD for the rouya Story FMV. 😆 That cutscene is arguably the roughest one.

Don't bother with the DVD upload. It's not worth it for me.

That's... really sad. 😢 I am going to send it to you anyways because it's certainly not a problem for me to do so. When I have the spare time, I like to help however I'm able.

NoUserNameForYou commented 1 year ago

...

Polymega commented 1 year ago

Here is the opening cinematic from the AoSH DVD, losslessly demuxed back to an MPG container: http://www.igotaletter.com/temp/VTS_01_3.VOB.mpg

Please let me know when you've downloaded it so I can remove it from my server. I'll remove it within about a day's time either way.

NoUserNameForYou commented 1 year ago

@Polymega

Please let me know when you've downloaded it

Thanks. I downloaded it.

I also created 60FPS files from YOUR files with RIFE. I screwed up big last time with lost details so nothing fancy like debanding, color grading or facial reconstruction etc. this time. Plus, Bink1 seems like it only does 16-235 (so when you adjusted the levels to 0-255 in Avisynth you screwed up as well).

I uploaded the bik1 files here. Three files with multiple mirrors

https://www.mirrored.to/files/1IPSMJ9H/movie1.7z_links https://www.mirrored.to/files/0OANA89K/movie2.7z_links https://www.mirrored.to/files/M1WPJS2H/movie3.7z_links

When the eventual MP4/FFMpeg file playback drops we should do the whole thing from scratch with correct color range leveling.

MarioTainaka commented 1 year ago

@Polymega

Please let me know when you've downloaded it

Thanks. I downloaded it.

I also created 60FPS files from YOUR files with RIFE. I screwed up big last time with lost details so nothing fancy like debanding, color grading or facial reconstruction etc. this time. Plus, Bink1 seems like it only does 16-235 (so when you adjusted the levels to 0-255 in Avisynth you screwed up as well).

I uploaded the bik1 files here. Three files with multiple mirrors

https://www.mirrored.to/files/1IPSMJ9H/movie1.7z_links https://www.mirrored.to/files/0OANA89K/movie2.7z_links https://www.mirrored.to/files/M1WPJS2H/movie3.7z_links

When the eventual MP4/FFMpeg file playback drops we should do the whole thing from scratch with correct color range leveling.

ooh! this is neat!

NoUserNameForYou commented 1 year ago

ooh! this is neat!

You're welcome. As for your previous suggestion, I don't think there's non-English dubbing for this game. Don't know about the PS3/Xbox360 remaster though... Even if there were, the new playback engine could use xHE-AAC (don't know about licensing) or Opus (free license) and they wouldn't make the file size that much bigger. 10 minute acceptable audio quality with xHE-AAC is just 5MB. Imagine four other languages, just 20 MBs addition.

MarioTainaka commented 1 year ago

ooh! this is neat!

YW. As for your previous suggestion, I don't think there's non-English dubbing for this game. Don't know about the PS3/Xbox360 remaster though... Even if there were, the new playback engine could use xHE-AAC (don't know about licensing) or Opus (free license) and they wouldn't make the file size that much bigger. 10 minute acceptable audio quality with xHE-AAC is just 5MB. Imagine four other languages, just 20 MBs addition.

There's currently a Brazilian Portuguese dub out there and it's really well done too. Isn't Opus open source and royalty free iirc?

NoUserNameForYou commented 1 year ago

Watched a few YT vids and damn, they did a good job. Maria could have been better though.

Dukiverse commented 1 year ago

I've always been interested in all work related to remastering/upscaling these FMVs. Even just for testing purposes, I have made many attempts on some of the videos but for the most part the results from the AI left me unimpressed. It is difficult to create much more information from what little the 512x384 videos provide. I decided to have a go at the deai FMV and experiment to get even better results. I started with the usual process of deinterlacing with QTGMC and keeping lossless image sequences between every export. This time, I utilized the scenes from the Art of Silent Hill DVD as it contained most of the content needed for deai along with snippets of the other movies as well. Working with the increased 720x480 resolution made quite a difference in comparison. I would say the FMV is 90% sourced from the DVD, with the rest having to be blended together from the slightly lower quality version. This didn't seem to work out too bad though, because the DVD contained most of the up-close detail shots and typically just cut away early when panning out or the last few seconds of Maria stepping off screen for example, so the fade to lower quality (still upscaled) does not look all that noticeable.

https://drive.google.com/file/d/1T6B_Tpp_Tc6NcHF1irh6NVnLTkF5AnQO/view?usp=sharing

Polymega commented 1 year ago

I really like the clarity of your deai example video here. 👍 It's brought a really nice level of sharpness without making things overall too sharp. Nice one! I noticed a couple of haloed edges from the sharpening, but by-and-large it wasn't present or noticeable. The two concerns I noted though are extreme ghosting during certain movements and the wobbly/spooky eyes that can happen from upscaling.

It is difficult to create much more information from what little the 512x384 videos provide. [...] Working with the increased 720x480 resolution made quite a difference in comparison.

💯 The jump from 512x384 to 720x480 doesn't seem like much, but yes, what a huge difference that makes for upscaling work. What I would give to have all the story FMVs at 720x480................

NoUserNameForYou commented 1 year ago

I'm currently playing the game in Reshade's superdepthVR with Virtual desktop SBS + Quest 2 and, what a nice experience. The grain kills the -ull encoding quality of VD but it's still manageable.

Need to convert the FMVs from 2D to 3D with MiDas + StereoPhoto Maker for future use.

Oh also, in psychology there's such concept called "disassociativity". I think when Konami censored the murder.bik and made it black and white as opposed to rest of the game, in-color they employed this phenomenon rather than just getting lower ESRB rating/no-controversy or higher profit.

They didn't want the player to feel THAT bad so that part of the FMV is different than the rest of the game and it immediately signals it to one's brain.

She wasn't even sedated when she was being murdered. She was vivid. Had to suffer through it all, guessing whether it was an act of mercy or apathy/hatred.

James kissed her forehead like a psychopath as if she was some kind of a ritual sacrfice and did his selfish deed.

IMO player had to feel like a POS after watching this since up until that moment he/she was empathasising with the protagonist.

Censored audio as well as B&W prevents that through disassociation. I'll be making my own deoldify'ed colored and uncensored version soon and will share it here. You need to realize what you have done in very graphic detail.

Polymega commented 1 year ago

If Gemini's MP4 player for the game comes to fruition, there's a real possibility that we can add in Mary's uncensored audio as a toggleable option. We have the means to add multiple audio tracks to an MP4, the same as Bink videos, and it can be a case of if VideoTapeFMV == active && UncensoredAudio == true, then play this track instead.

NoUserNameForYou commented 1 year ago

If Gemini's MP4 player for the game comes to fruition

If not, Bink2+patched binkw32.dll works with the EE game as well. You need to refactor some code around because one of the settings causes black screen during FMVs. When it's turned off, bik2 files play and they also have capability for 0-255 levels and 10bit.

MarioTainaka commented 1 year ago

If Gemini's MP4 player for the game comes to fruition

If not, Bink2+patched binkw32.dll works with the EE game as well. You need to refactor some code around because one of the settings causes black screen during FMVs. When it's turned off, bik2 files play and they also have capability for 0-255 levels and 10bit.

There's one issue with using bink 2... there are no Free tools available for it

Polymega commented 1 year ago

There's one issue with using bink 2... there are no Free tools available for it

Yep. Also, Gemini is a big advocate of MP4 > Bink (1 or 2) and I'm in agreement with him about that.

MarioTainaka commented 1 year ago

There's one issue with using bink 2... there are no Free tools available for it

Yep. Also, Gemini is a big advocate of MP4 > Bink (1 or 2) and I'm in agreement with him about that.

There's one concern I do have with using MP4.. how will it be decoded? will it be using the GPU or CPU to do the decoding?

Polymega commented 1 year ago

Gemini would have to answer that. He's currently taking a break from this as, if he were to implement such a feature, he'd also want his code to handle the Bink 1 video files (if the player isn't using a FMV Pack with MP4s). Ideally, he'd want his solution to handle both formats and throw the game's original binkw32.dll out the window. The problem is, documentation on the Bink format is sparse so he's ran into a few frustrating problems with decoding the format.

NoUserNameForYou commented 1 year ago

@MarioTainaka

There's one issue with using bink 2... there are no Free tools available for it

@Polymega Would we have any legal problems using the Unreal Engine's bik2 encoder? Bik2 encoder files are freely distributed with UE install (and they are also on Github), unfortunately I don't know the policies on whether one can use the files in a non-Unreal game...

GPU or CPU to do the decoding?

I'm quite sure he'll use FFMpeg (without Fraunhofer AAC) as I don't see any other way of doing it. So it'll have GPU acceleration though he might switch it off because of tech difficulties like color shifting etc.

he'd want his solution to handle both formats and throw the game's original binkw32.dll out the window.

FFMpeg can play both audio and video of bik1.

NoUserNameForYou commented 1 year ago

For those interested, 60FPS stroy FMVs in 3D stereo Half SBS format

https://www.mirrored.to/files/6HDVHNK9/movie3d1.7z_links https://www.mirrored.to/files/BUVYMBE6/movie3d2.7z_links

Turn off Reshade's Superdepth3D or your 2D to 3D conversion software during story FMVs and let the headset/TV handle 3D.

TommvinHuwaltzky commented 1 year ago

There's currently a Brazilian-Portuguese dub out there and it's really well done too.

There shouldn't be any dubbing, and the SH2EE-team should restrict the usage. Subtitles yeah, but dubs, never.

It was the original intend of Japanese games from back then. Resident Evil, Silent Hill, etc.... Even the original Japanese versions of the games only had subtitles. It was supposed to be "American horror through a Japanese filter". The voice-actors/actresses were selected special in Japan, and then worked with Team Silent. There were no dubs allowed on "foreign" versions of the games on purpose. So that original intent should be respected, just like any other aspect of the game the team tries to retain as original as possible (:

Bellamaia commented 1 year ago

Forgive my intrusion, but many agree with you, as do I. I'm on several Brazilian Silent Hill servers on Discord, in addition to following the most relevant influencers for the franchise in Brazil, practically all of them don't use fan dubs, they prefer the originals, in addition, recently, the SH2EE project presented us with a translation in Impeccable Brazilian Portuguese, it even looks like it was made by Team Silent itself, the level of quality is professional! All servers think so and loved the translation, in addition to considering it the best work ever done in the genre here in Brazil, so in fact there is no need for this dubbing.

ElTioRata commented 1 year ago

Just read subtitles faster smh

justtoask commented 1 year ago

Hey, could you reupload these videos somewhere else, like pcgamingwiki, or archive.org at least? The uploads available here are very slow.

NoUserNameForYou commented 1 year ago

Hey, could you reupload these videos somewhere else, like pcgamingwiki, or archive.org at least? The uploads available here are very slow.

Unfortunately no, as I have slow internet. But the download speeds shouldn't be "complainable slow"? I mean 1GB download shouldn't take more than 10mins on those free services, if you were getting too slow downloads, try some other time. They may be congested. In my case, bayfiles with IDM/jdownload is fast enough.

justtoask commented 1 year ago

@NoUserNameForYou Can you upload the files again? All the links are dead.

NoUserNameForYou commented 2 months ago

Hi there

Hello @Polymega is there any way I can contact you privately?