search

07th-mod / umineko-question

93 stars 9 forks source link

Patch thread #1

Open ItaloKnox opened 8 years ago

ItaloKnox commented 8 years ago

Requesting you here, @enumag

Let me make a full breakdown about my findings:

The full original voiced script can be found here. Original file format was a compressed .dat file, but this is irrelevant since the new script follows a different format.

New script is here. Can be read without any workarounds. File needs to be .utf. Engine's manual is here.

For now the only thing we need to do is to put the voice files on the script.

Original script excerpt:

;「…屋敷内の戸締りは厳重に確認してきました。@ですが、万が一ということもあります。@皆、なるべくここに一緒にいた方がいいでしょう。」@
nat:dwave 0, nat_1e323`"...I carefully checked all of the doors and windows throughout the mansion.`@dwave 0, nat_1e324`  However, we must be prepared for anything.`@dwave 0, nat_1e325`  Everyone, I think it would be best if we all remained here."`\ 
E_N
ld l,$EVA_AkireA1,23

New script:

langjp「…屋敷内の戸締りは厳重に確認してきました。@ですが、万が一ということもあります。@皆、なるべくここに一緒にいた方がいいでしょう。」@
langen^"...I carefully checked all of the doors and windows throughout the mansion.^@^  However, we must be prepared for anything.^@^  Everyone, I think it would be best if we all remained here."^@
ld l,$EVA_AkireA1,23

Now two scripts that doesn't match.

Original:

;　私にとって魔法が存在しないのは、私の世界の話。@
;　だからといって、お姉ちゃんの世界までを否定するのは、まったくのお門違いなのだ。\
`Magic not existing in my eyes...matters in my world.`@
`Denying it even in Onee-chan's world just because of that was totally barking up the wrong tree.`\

New:

langjp　私にとって魔法が存在しないのは、私の世界の話。@
langjp　だからといって、お姉ちゃんの世界までを否定するのは、まったくのお門違いなのだ。\
langen^The fact that magic doesn't exist in my eyes...matters in my world.^@
langen^But denying it even in Onee-chan's world for that reason was totally misguided.^\

Looks like a really shitty job to do. Any ideas?

Forteissimo commented 8 years ago

Sorry, I don't have the full-sized backgrounds renamed, but I'll try to do them as soon as I have the chance.

ItaloKnox commented 8 years ago

Hello, guys, just doing the final changes and updating the documentation so we can release the first stable patch.

There are a few issues I still need to fix, though, and the biggest of them is the fact that I can't get @drojf's script to work. Everything is in place, but I receive an error when running it:

W:\Umineko1to4>python patchUminekoToWidescreen.py
  File "patchUminekoToWidescreen.py", line 97
    def modifySetWindowArguments(setWindowArguments : str):
                                                    ^
SyntaxError: invalid syntax

After fixing that, the last thing to do is to put the full-sized backgrounds in the game and release it. Then we can focus on fixing the menus and inserting the CGs and everything will be finally done.

I have all backgrounds (I guess) in my OneDrive, so if anyone wants to download it, go ahead. If I recall correctly, they're all properly named, so it's just a matter of running a folder diff to delete the extras, then replacing the folders. I'll be doing it tomorrow so I can report the results. If everything is ok, I will just compile a new .nsa remotely for the sake of saving time and release the patch. Might take a day or two, but we're getting there.

drojf commented 8 years ago

I think you're using python 2.5?

Just change

def modifySetWindowArguments(setWindowArguments: str):

to

def modifySetWindowArguments(setWindowArguments):

I haven't tested with python 2.5 yet though, there may be other problems. You can just download python 3.5 instead if there are any more problems.

Just for fun i was working on something to play the original ps3 .at3 voice files, but honestly it doesn't make much difference. I just know that umitweak's voice files (ogg) are encoded in stereo, so half of the bandwidth is being wasted. But even when you compare it to the original at3 files, it's quite hard to tell the difference. Don't worry about it for now lol I don't even know why I'm mentioning it.

ItaloKnox commented 8 years ago

Oh, yeah, I guess that's it. I never used Python 3.5 at all, so I'm not used to the syntax changes. It's working like a charm.

If I recall correctly there was something like that back when the game was released. I wish I had the tools stored somewhere because it would save a lot of time and cover a few missing stuff, but it's no big deal.

I think we don't have to improve in audio unless we rework the entire game, which is not the ideal scenario. In this case, there's Umineko Project already. If MangaGamer released the game as an Unity game like they did with Higurashi, with a little work we could have made a light version of Umineko Project.

Anyway, just got everything fixed, now I have to setup the backgrounds and then repeat the whole process on my server. Hopefully I'll have it done today.

drojf commented 8 years ago

If you need help with anything for the release let me know. And remember that you shouldn't pass the "0.utf" file through my script twice. The script adds an offset and scale to the parameters to the setWindow function's original arguments, so running it twice will have the wrong effect. Therefore it would be a good idea to keep an 'original' version of the script and a 'widescreen' version of the script in case you want to make changes later to the 'original' version.

I think the background images should be 1706 x 960 - if they are any larger the game will display them zoomed in! Please check this.

Also, if you're re-doing the backgrounds, you can try making a version with PNG files renamed as .bmp files to get the size down - afaik the game still accepts and displays them. I guess this isn't really an issue for distribution size because the .bmp files get compressed anyway.

the below is not important

all the (variable_name : str) does is tell the IDE what type the variable should be for static analysis, I don't think it's even used when the python code actually executes, just that the newer python 'accepts' it as part of the grammar.

ah, what i meant was that ffmpeg based media players can play/read .at3 files natively, so I was making a program which intercepts when the game accesses the .ogg voice files, and plays the corresponding .at3 files in a background thread/program (easier than it sounds). I just didn't like the idea of using doubly encoded files, especially when the original files were only encoded 64kB/s, but in the end the difference is very little. I'm not in any way saying you should add this to the release, I might release it myself as an add on or something if people want to try it. I talked to the umineko project people before, and at the time they were using high quality .ogg files for voice playback, not the originals.

ItaloKnox commented 8 years ago

Yeah, actually I just passed a "dummy" and replaced it with my own 0.utf. From what I can see, it's working like a charm. I tried a couple of backgrounds in their original resolution and they looked fine. I need to double check this, btw.

Anyway, I'm having some issues with the backgrounds. It's actually not about making them work, but more like having a terrible internet speed to upload the manual changes. I usually work with a remote Windows on Azure, but in this case there are a lot of steps to replicate, making it too convoluted to properly test and release on a remote machine...

So, anyone willing to take over and work on the backgrounds? That's pretty much the last thing to do for a while, after that what need to be changed is pretty simple and small, just a little time consuming. I made this little script to move all widescreen backgrounds with a filename match to another folder, so the only thing you need to do is to convert them to bmp (or we can just edit the script to work with pngs) and place them on the right folders (airport, aquarium, etc). After that it's just a matter of checking each moved background to see if they are "complete" (the filenames will always match but some PS3 backgrounds have transparent areas, so you need to find and replace them with the complete background - sometimes they're next to each other or with the leftovers).

Sounds confusing but I couldn't find a way of properly explaining that. Anyway, I'm doing some stuff with Higurashi right now, so if no one wants to do it I'll just come back after I finish what's pending with Higurashi.

drojf commented 8 years ago

hmm I'll have a try at it. I'll probably make a script for it (including resizing/merging the transparentimages) in case we need to change something later/to keep track of what changes have been made. I should be able to upload it ... slowly (by my calculations I can upload at 1 gigabyte / 3 hours),

I'll use Forteissimo's work as a base to compare the images to see if I'm doing it "correctly" (makes the task much easier since I don't need to play the game to check). I tried out doing "airport" and it doesn't seem too bad (by "doing" i mean writing down the mapping from image->image in Python for only the images which don't share identical names).

I definitely checked the images in-game and they're zoomed in, but only by a moderate amount (2366/1706 = 138% zoomed in), but you can't tell at first glance.

drojf commented 8 years ago

Just checking in, I ended up using image hashing to figure out which images fortissimo used from the ps3 assets. I did try just by doing 'closest name match' but it doesn't do a very good job compared to either Umitweak or Fortissimo (assuming they've done it correctly that is).

There are only about 20 it couldn't recognize, 10 of them because he used the original game's images, and the rest are more or less special cases. I also had a look at what Umitweak had done for those missing images, and they seem to have just chosen the most similar ps3 image they could find rather than use the original low res images.

https://docs.google.com/spreadsheets/d/1-qODgbEt8vy2Wvy-lSYCgHjq1bLXGaW1qxW29s5Jr2U/edit?usp=sharing

Will probably have it done by this weekend. Just need to fix the 3 or so images which are special, then resize all the images to 1707x960, and then upload it.

drojf commented 8 years ago

Alright, files are uploaded. Please use 7-Zip to extract the .wim files: MEGA link

So a few things regarding this:

ItaloKnox commented 8 years ago

I should have released the patch a few days ago, but for some reason (mainly Overwatch?) I didn't.

Anyway, v1.0 is finally released. :) Many thanks to everyone in this thread and sorry about the delays. For now we can say the patch is fully playable, but there's still a few things left to fix and add. I plan on doing them manually later, maybe after finishing the graphics patch for Higurashi Tatarigoroshi and the voice patch for Higurashi Himatsubushi. Most likely sometime next month, being optimistic.

So, again, thanks for all the help and please tell me if I forgot about anything.

dpgagnon commented 8 years ago

Thank you for all your hard work everyone! I for one very much look forward to playing through with the patch :)

DoctorDiablo commented 8 years ago

Someone found an issue where achievements are being granted at the wrong time. Looks like Mangagamer released a patch that fixed this a while back, so I merged those changes into the latest 0.utf. It includes some minor spelling and grammar fixes too.

MG also changed Bern's "Mii" to "Meep" which is really dumb, so I left that part out. It can be changed if anyone really minds.

UzukiChigun commented 8 years ago

Greetings. Sorry, just an outsider stopping in to say thank you to everyone who brought this patch to fruition. I do have one question: is it possible to apply only voices to the steam version without making any graphic-related changes? If there is a way to make this work I would appreciate any direction that can be provided. If not, oh well. Anyway, thanks again!

drojf commented 8 years ago

I think if you download this version of the 0.utf:

https://github.com/07th-mod/umineko-question/commit/ff5cd239c514fb33ca610c79ef76ab3fcf1d6beb (last version of 0.utf before we started messing with the backgrounds)

then it would work. Just download the voice files in the instructions, and the above special 0.utf, and do not patch anything else. It's also advisable to change the following lines in the script:

umalias default_vol, 70

change the above line to:

umalias default_vol, 15 voicevol 20

as shown being changed here: https://github.com/07th-mod/umineko-question/commit/dcfa55ffbc9831dc07d00a8dd3098d74e27dd1a9 (version with music/voice volume adjusted)

This sets the music volume, and the voice volume (set it how you wish).

....I think that would work, let me know how it goes.

UzukiChigun commented 8 years ago

Works like a charm. Thanks a bunch for your super quick reply and solution!

Maelstrom1 commented 8 years ago
umineko glitch

I think I found a graphical glitch in EP1. The background should be the bedroom of the guesthouse, when Maria goes in to wake up the cousins on the 2nd day. However, this background appeared for some reason halfway through the dialogue. Note that, during the first line, it was fine. After this, the screen went to black, as I think it should have, before returning to the proper background.

drojf commented 8 years ago

So i looked up that part in the script, copied it to the top of my script, and played it, and confirmed this happens.

It looks to be this part

langen^When she had left the room, her face had been sleepy, but now that she was back, she looked irritated.^@ bg M2F_R1c,24

The image correctly matches both fortissimo's chosen image and the old umitweak version. In the original 'old' images, it appears to be part of the same room as m2f_r1a, m2f_r2a, m2f_r3a. The images look like this: http://imgur.com/a/Kozrc

Strangely, this is the ONLY place this image (M2F_R1c) is used in the entire script.

I think there are two things it could be

  1. it was meant to show 'not the bed' or 'something else', for comedic effect. We should use the original image in this case
  2. it's meant to be G2F_R1c (picture of the bed) and is just an error.

So either we change this image to the original (http://imgur.com/nAzLGny), or we edit the script to show G2F_R1c (a picture of the bed). I'd probably go with number 2 otherwise we're forced to use the old image.

Anyone have the original script (umitweak or even older comiket-on-a-cd version) to check if this was a bug introduced in the manga gamers / Steam version or not? Then we can tell if it was intentional.

DoctorDiablo commented 8 years ago

This is the old umitweak version of the script, and it seems to be the same way there.

I'm pretty sure it's supposed to be G2FR1c because all the other bg commands around there start with the G2F prefix.

SomeGuy87 commented 8 years ago

Very nice work guys! Just bought the game yesterday and started reading it with this patch - really makes a big difference quality-wise, especially the backgrounds.

Small issue I experienced so far: It seems like Krauss' image switches back and forth between your version and the one from Steam depending on his pose, e.g. during his encounter with Kinzo.

http://imgur.com/a/OFYCz

Additionally, there is a volume issue whenever multiple voices are played at the same time. One of the samples seems to be twice as loud.

Maelstrom1 commented 8 years ago

Can confirm. Forgot to mention that. In addition, Idk if it was part of the new translation, but a portion of dialogue is missing from when, on night 1, Shannon walks back to the guesthouse with the cousins and a sleeping maria. It skips right from them walking to Batler saying how he wishes he was holding maria. (b/c shannon as a little too close to george). Oh, and, again, idk if it's purposeful or not, but some of Battler's lines are just straight-up missing spaces. Idk if either of these are Steam version changes, or unintended consequences.

drojf commented 8 years ago

I think it'd be a good idea to issue bug reports as separate issues (as in, not all in this huge thread) so we can keep track of them. @ChrisE87 and @Maelstrom1 - could you do me a favor and make new issue by copying your comment text as a new issue? Same goes for anyone else who has bugs.

krauss sprites: see https://github.com/07th-mod/umineko-question/issues/12

regarding the voice playback: see https://github.com/07th-mod/umineko-question/issues/13

drojf commented 8 years ago

@Maelstrom1 ah bad comment timing.

Could you raise those as a new issues? So that we know where in the script the problem is, could you also copy some of the text from the game (a sentence would be fine), so we can search for that line of text in the script.

dpgagnon commented 8 years ago

MG updated the game on Steam today which broke the patch since it replaced both the exe and 0.utf. I have no idea what they changed.

I'm just using the old exe and utf for now

As far as volume is concerned I cranked the voicevol all the way to 100 because it's super quiet by default and was getting drowned out by the rain sfx and bgm very frequently. I've got default_vol at 25 but that seems to be kind of spotty since the game will change volume all the time for effect.

Maelstrom1 commented 8 years ago

Wtf. The updated .exe file did some crazy crap. My most recent save data warped me to a death scene in an episode I hadn't got to yet, and all the text ran off the side of the screen. I'll try to stick the old .exe back in and stuff and see if that works. Oh, and the sound on this new .exe is absurdly loud. Will edit this post with more info, unless someone posts first.

edit: Works fine when everything was replaced.

drojf commented 8 years ago

Probably when you went to load your old save, it would try to load the old line number in the updated script. Because we added in so many voice lines, it would shift you forward in the script by some amount.

Does steam actively delete files added to the game folder when the game updates? If it doesn't, we could fix this by using a different exe file name and telling it to load a different script by command line arguments (eg Umineko-patched.exe -script patched_0.utf). Of course, you could also just move the entire game folder somewhere else to prevent updates since there's no DRM (I think), but that would break steam achievements.

DoctorDiablo commented 8 years ago

Really surprised they bothered to release an update. MG released an update in July a week after release, and haven't done anything since then.

No idea if they changed anything in the exe, but I diffed the script and only found 2 changes:

langen^...Right there, a handle with a demon-shaped design was buried into his chest.^@ langen^The same type of weapon that had been stabbed into Eva and Hideyoshi's foreheads...was in Kanon's chest...^\

changed to

langen^...Right there...was a handle with a demonic design engraved in it...^@ langen^As Kanon looked down at his chest, the same type of weapon that had been stabbed into Eva and Hideyoshi's foreheads was...^\

and

langen^"Yeah, ...that's right...^@^ ...Mama was just...being possessed by the evil witch again...^@^ ...I'm sorry, ...I'm so sorry..."^\

changed to

langen^"Yeah, ...that's right...^@^ ...Mama was just...being possessed by the bad witch again...^@^ ...I'm sorry, ...I'm so sorry..."^\

SomeGuy87 commented 8 years ago

Why on earth do they release a patch for those two changes? They really want to make your life hard, don't they? :D

Is there any possibility of changes like those corrupting saves unrecoverably? I could imagine that at some point, this patch needs to be applied to the newest files, which would leave the saves in an inconsistent state for the new ones, right? How is this handled in the non-patched version, are all saves modified along with updates? I mean if they script gets longer for some reason, the saves shouldn't work either...

marakeusz commented 8 years ago

For anyone who looks into the script from time to time it is no problem to jump from one part of the story to another (and make save file of that), but everyone else are screwed.

As long as developers know about possible problems - I think we can assume no massive script update will appear. At least we can hope.

EDIT: The Steam users at least are getting update only of particular file. The owners of MG drm-free version (such as me) must download 2.6GB zip file if they want to have most updated version...

ItaloKnox commented 8 years ago

Hey guys, just a heads up about development.

I've noticed there's a lot of issues to fix and it's great that we have so many dedicated people helping to solve them. If you don't follow Higurashi's development, the reason why I'm not always present here is because we are still working on the graphics of Tatarigoroshi and Himatsubushi was just released. Because of that, I can't push updates in a timely manner, so I invited @drojf and @DoctorDiablo to the organization because you guys are always helping out a lot since we started. I still read all issues and try to answer them when needed, but I'm giving you guys full access to the repo so you can release updates when needed. You shouldn't worry about pushing updates on my instead, it's just a way of making sure someone else can push updates in case we find a game breaking issue and I'm not here to draft a new version.

Anyway, great work everyone and keep it up. 👍

ReitoKanzaki commented 7 years ago

I thought I'd do at least something to contribute. I got a little tired of the original videos and how they did not match the sprites and were of low quality.

I managed to find a video that was OP3 redone with the PS3 sprites. I downloaded it and tried to load it into the game after converting it to MPEG1, which is apparently the only thing PONScripter can play, the window would only show the upper left portion of the screen. I ended up converting it to 480p as that was the only way to get the video to show up correctly. Please note that there is a border of 10 pixels on right. I had to pad this so TMPGEnc would not yell at me for not having mod16 dimensions. It is invisible when playing the video in game.

Using the same technique, I encoded the PS3 OP to replace OP1.

If anyone knows how to get HD videos and/or modern codecs to play, I will encode the videos again so they will look nicer. Not a priority, but I wracked my brain trying to get it to work.

https://mega.nz/#!okMHUQRC!Dt_HJw9C9O5hvfLChniovKm7Y7OgkWp7YiSjtR_4X8E

PS: OP2 appears to be WMV9+MP3 wrapped in AVI. PPS: Apparently you can add soft subtitles to the videos. But I cannot figure out how to do it without crashing PONScripter.

ReitoKanzaki commented 7 years ago

Hey guys. I finally finished playing the game last night. It was absolutely enthralling. It did take me over 90 hours to complete it (I try to listen to all the voices without skipping them).

After making the videos, I decided to try and tackle some of the static effects that still used the original graphics. Specifically the opening sequences, Beatrice vs. Virgillia, and Beatrice vs. Battler. There were also some graphics that were not converted to widescreen. All of these used assets from the PS3 version as either CGs or screenshots taken from the videos. Enjoy.

effects.7z

I'll try taking a stab at the dynamic effects, such as the ending screens, and restoring/replacing/adding the CGs. Chances are this will require editing the script.

DoctorDiablo commented 7 years ago

@ReitoKanzaki If you're serious about adding in the CGs, awesome. Here are some tips.

All the CGs should already be in the big\bmp\background\cg folder. To get them to display, you have to update the script file 0.utf. You should be able to display them the same way that the backgrounds are drawn, using the bg command like this:

bg air_o1a,0

air_o1a is a stralias defined near the top of the script that maps the alias to the full file path of the image. 0 is the effect the image is drawn with (fade in from right/left, etc). Those are also defined near the top of the file, and I think 0 means draw with no effect.

During the time that the CG is displayed, you'll want to prevent the script from rendering any character sprites on top of it. Sprites are drawn with the ld command, so you can just comment out the ld lines by putting a ; in the front of them.

ReitoKanzaki commented 7 years ago

@DoctorDiablo Thanks for the tips.

I found at the bottom of the script a function already in place that already will display the CGs.

set_oldnew_bg $VAR,$VAR,%VAR,%VAR

After thoroughly tracing it, it will apply the CG only if a global variable is defined. It also automatically hides any sprites until clr_oldnew_bg is called. I forced it on and it works.

I'm trying to add in the PS3 CGs using this method.