rufaswan / Web2D_Games

2D Games playable on the web!
https://rufaswan.github.io/Web2D_Games/
GNU General Public License v3.0
24 stars 6 forks source link

about switch vanillaware games .mbs file convert #12

Open Cloud2333 opened 6 months ago

Cloud2333 commented 6 months ago

Hi, rufaswan! I'm sorry to bother you, I followed the tutorial to try to convert the file, but the .quad seems to have some problems and is a little different from the example you sent. The converted .quad file also won't play properly. I don't know which of my steps is wrong so I want to ask for help. QQ截图20240310010404 Here is a screenshot of my steps. QQ截图20240310010811 Here is the file I converted. https://mega.nz/file/rBdx1YII#GAsi2bebJkjMXhMiwIi2KeLxMY3Tcn_FQOLmUACULLI I'm not good at English, please forgive me.😵

rufaswan commented 6 months ago

It is great you uploaded the .v55 and .prediv.quad along, I can compare them step-by-step along.

The .v55 file is same as mine, so "quad_vanillaware_FMBP_FMBS.php" step is not the problem.

The .prediv.quad is different. Starting from "animation_timeline" object (line 119913), the "matrix" should be identity matrix, but you got a all zero matrix instead. Scale X and Y should be 1, but yours is 0.

Tested with PHP 7.0 to 8.2, I couldn't reproduce the same .prediv.quad as yours. It is working fine for me.

Can you tell me more about your PHP? Like version? 32-bit/64-bit? Did you change anything other than "memory_limit" on php.ini? Thanks!

Cloud2333 commented 6 months ago

Sorry for the late reply. In order to confirm the problem, I also conducted some tests. PHP 7.0 to 7.4 and PHP 7.4 to 8.2 running logs are different, but I get the same .prediv.quad. The .prediv.quad is the same whether it is 32bit or 64bit. php.ini I only modified the "memory_limit" to 2048M. In addition to "Unicorn Overlord", I also tested "Grim Grimoire OnceMore" and "13 Sentinels" and got some interesting running logs. Therefore, I have prepared the test files and running logs of versions 7.3.9 and 8.1.20, hoping to help you confirm the problem. https://mega.nz/file/KNN3ESSC#_UARe-T67Ab9Rm-l6p1GNOIMZ32BznnP4ssiqwVlwts

Cloud2333 commented 6 months ago

By the way, I borrowed my classmate’s computer for testing, and the final converted .prediv.quad were still the same. We all use windows.

Cloud2333 commented 6 months ago

I tried to use a virtual machine to run Linux and convert the files in the Linux environment, but the final files were still the same as before.😖 But there's other good news, I got the files for the PS4 version of Unicorn Overlord. The download link of the file has been sent to your email. If you are interested, you can confirm it.
Thanks again for all your hard work! 🫡

rufaswan commented 6 months ago

Thanks! According to your log file, this warning caught my attention

Warning: Trying to access array offset on value of type int in F:\unpack tools\psxtools\common-quad.inc on line 263

This file handles matrix, so it is the reason why your .prediv.quad is different. It seems the reason is the matrix (an 16 number array) was converted into an int for some unknown reason.

I couldn't find anything wrong with it, other than variable is in uppercase, $M1 and $M2. Can you test with this new PHP script, and see if you get the same expected output?

php  php_uppercase.php

You should get 4 strings and 4 arrays, like the comment within the script. If you got an int somewhere, let me know.

https://github.com/rufaswan/Web2D_Games/blob/master/tools/php_uppercase.php

I also "fixed" for quad_vanillaware_v55.php , but I have no idea if it will actually works. You can also give it a test.

Thanks again for helping debugging a weird bug!

Cloud2333 commented 6 months ago

"php_uppercase.php" should work correctly, here is the screenshot. QQ截图20240311222539 I've tested all three games, this time the conversion got .prediv.quad is different from before, the final conversion got .quad can also be played normally using player. QQ截图20240311225300 In fact, I also tested “Odin Sphere”, “Dragon’s Crown” and “13 Sentinels” on PS4 platform. Only "13 Sentinels" had problems converting files, the other two games ended up with files that played correctly. This time the problem occurs in "quad_vanillaware_FMBP_FMBS.php". QQ截图20240311234217 The logs and.mbs files are here. https://mega.nz/file/aUE1wTgK#1ECuE6muwoi3eN5Wnts6k2mmUOODdQ2E9mXai0aPmKk I think this is the last problem I have, thank you very much!🫠

rufaswan commented 6 months ago

Great! I was correct about these uppercase variable name are the problem, but I couldn't exactly nail down the exact step to reproduce it.

Can you run "php_uppercase.php" one more time, please?

Also, "quad_vanillaware_FMBP_FMBS.php" is updated with fix for 13 Sentinels, and added game_id "ps4_unic" for PS4 Unicorn Overlord support.

Let me know how that goes. Thanks!

Cloud2333 commented 6 months ago

Regarding "Unicorn Overlord", I just tried to use the switch version's .mbs conversion solution to convert the PS4 version's .mbs file, and the file can be used normally. However, the latest QUAD PLAYER seems to be unable to load multiple textures at the same time, but the old version of QUAD PLAYER can load textures correctly and play them. QQ截图20240312003056 QQ截图20240312003311 QQ截图20240312003319 QQ截图20240312003406 But when exporting images, the exported "skeleton" types are all 0kb, and only the uncombined "animation" types can be exported. I'm a little concerned about whether it's a BUG, ​​so I'm asking by the way. Sorry to bother you again!🫠

Cloud2333 commented 6 months ago

This is the result of running the latest "php_uppercase.php", I hope it can help you!🫡 F:\unpack tools\psxtools>php php_uppercase.php string(21) "function lower_case()" string(21) "function mixed_CASE()" string(21) "function mixed_CASE()" string(21) "function UPPER_CASE()" array(3) { [0]=> string(1) "a" [1]=> string(1) "b" [2]=> string(1) "c" } array(3) { [0]=> string(1) "d" [1]=> string(1) "e" [2]=> string(1) "f" } array(3) { [0]=> string(1) "g" [1]=> string(1) "h" [2]=> string(1) "i" } array(3) { [0]=> string(1) "j" [1]=> string(1) "k" [2]=> string(1) "l" } == test_func() array(3) { [0]=> string(1) "a" [1]=> string(1) "b" [2]=> string(1) "c" } array(3) { [0]=> string(1) "a" [1]=> string(1) "b" [2]=> string(1) "c" } array(3) { [0]=> string(1) "a" [1]=> string(1) "b" [2]=> string(1) "c" } array(3) { [0]=> string(1) "a" [1]=> string(1) "b" [2]=> string(1) "c" } == test_func() array(3) { [0]=> int(0) [1]=> int(1) [2]=> int(2) } array(3) { [0]=> int(0) [1]=> int(1) [2]=> int(2) } array(3) { [0]=> int(0) [1]=> int(1) [2]=> int(2) } array(3) { [0]=> int(0) [1]=> int(1) [2]=> int(2) }

Cloud2333 commented 6 months ago

The latest version of "quad_vanillaware_FMBP_FMBS.php" can be used normally for 13 Sentinels, but there seems to be some minor issues for PS4 Unicorn Overlord. QQ截图20240312011804 Although I'm not good at PHP at all, I think this can be easily fixed, thanks for letting me test it! 🫡

rufaswan commented 6 months ago

Yes, you can use "swi_unic", but the .quad file will be tagged with System = Nintendo Switch. It is mainly to prevent PS4/PS5/Xbox confusion later on, if they have any difference.

I think WebGL run out of memory when exporting, so the files are all 0kb. Combining the fact Chrome is memory hungry, and with multiple 4K HD textures, WebGL just don't have enough memory to create new texture for export.

rufaswan commented 6 months ago

I really unable to understand why uppercase variable name will be converted to int for no reason, and the output "php_uppercase.php" doesn't have any problem, so I couldn't reproduce the error either.

What an weird error... :(

Cloud2333 commented 6 months ago

I get it, thank you very much for answering my question!

rufaswan commented 6 months ago

Fixed the "ps4_unic" tag not found error.

Cloud2333 commented 6 months ago

It works normally this time, thank you for your hard work! 🫡

rufaswan commented 6 months ago

Late here. I'll debug that multiple texture upload error tomorrow.

At the mean time, have fun!

Cloud2333 commented 6 months ago

It's really fun to see Galerius's sprites in action, thank you so much for the tool! 😊 1111 I also tested some special characters and found some interesting things. 🤔 QQ截图20240312160550 000000 This time there are many characters sharing one body. If played with QUAD PLAYER, the heads of all characters will be loaded at the same time.The head image layer of some characters has a higher priority. In order to confirm what happened, I chose a texture that was relatively easy to analyze and used PS to erase part of texture. Here are some test files that might help you improve QUAD PLAYER. 🫡 https://mega.nz/file/PR9X1BoY#EMrRGXB0EHZcofBwz45AIeYq6_6crADtviu-zON0s9U

rufaswan commented 6 months ago

The multiple texture error for QUAD PLAYER is fixed.

Cloud2333 commented 6 months ago

There seems to be a more serious problem.But this is only the case with EDGE, Firefox works fine, it's really strange.🤔 QQ截图20240312200458 QQ截图20240312201130

rufaswan commented 6 months ago

Interesting! In older Grand Knights History (on PSP), the head and the body are 2 different .quad files. So the whole sprite is created by merging those .quad files. And then there are accessory, like earrings and helmet, also on its own .quad file...

https://www.vg-resource.com/thread-38430-post-666571.html#pid666571

Looking and comparing how many parts are removed, there seems to be a new "set_id" field that was not existed on older vanillaware games. I'll need more info for debugging this new field. Do you know how many varieties Black Knight has?

rufaswan commented 6 months ago

The new QUAD PLAYER version is 2024-3-11

rufaswan commented 6 months ago

It seems EDGE max texture size is 4096 (or 4K HD), while Firefox max texture size is 8192 (or 8K HD).

Cloud2333 commented 6 months ago

Oops, I accidentally made a mistake, Firefox works fine too. QQ截图20240312201620

Cloud2333 commented 6 months ago

There are only 2 types of head parts in the Black_Knight_M00 image, and the Feather_Shield_M00 also only has 2 types of head parts. However, the game seems to have multiple characters set up by changing the colors. As for the specific number of characters in the game, I am not sure.

Cloud2333 commented 6 months ago

Black Knight shows in the game's Classpedia that there are only 2 varieties, which may be useful information. QQ截图20240313001612 Black Knight HG is shown in the game's Classpedia as having 3 varieties. QQ截图20240313001940

rufaswan commented 6 months ago

Just realized you can customize all your characters. You'll need to get "Idealist's Handmirror" (cost 30000 G) from the Black Market Dealer.

https://m.youtube.com/watch?v=Ta070xCrw2A

Cloud2333 commented 6 months ago

Yes, this time you can change the color of some parts of the character. They also set up different characters of the same class by changing colors and setting different heads.

Cloud2333 commented 6 months ago

During the development of the game, the development team also considered setting up a character's costume destruction system, but gave up in the end because it was too complicated. Probably because of the heavy workload, they didn't make textures for different weapons. This game took about 10 years from planning to official release.🧐

rufaswan commented 6 months ago

It reminds me of Dragon's Crown "mix-and-match" update, where the characters have 4 different texture for 4 different color outfit. The weapons also are all different textures, but all uses the same texture UV position. That's how the weapons can be swapped. This allows the character in any color to be paired with any weapon in the game.

Seems like Vanillaware managed to create a system to merge all that into one. Will need more info and time to research about it.

Thanks for this new important discovery!

Cloud2333 commented 6 months ago

Actually, I discovered something interesting again. QUAD PLAYER seems to have mistakenly recognized the brightness of the texture in the game as the transparency. The darker the texture, the higher the transparency.(Galerius' ICON actually has this problem too) 未标题-1 1111333 未标题1-1 And they seem to have used some special methods to achieve the edge lighting effect of the character and background objects. QQ截图20240314011035 (I would like to express my sincere thanks to Galerius for his contribution to the investigation. 😂) I think this is probably one of the reasons why they did not create weapon textures. Different weapons also need to set different parameters to ensure the coordination of the picture.🤔

rufaswan commented 6 months ago

Those transparency errors are from blending texture with vertex colors. Maybe Vanillaware changed it to mean something else since they no longer use vertex color only for parts?

As for the edge lighting effect , they are called "Rim light", a cinematography and photography effect. Sprites being 2D images can also utilize them.

https://www.studiobinder.com/blog/what-is-a-rim-light-photography-definition/

You can imagine how deep the rabbit hole goes. That's what I'm researching.

I've sent an email to you. Hope you can join us in discovering more things. :)

Cloud2333 commented 6 months ago

I would love to join you and help improve QUAD PLAYER, I actually came across version 1.0 of QUAD PLAYER a few years ago while investigating Azure Striker Gunvolt. 😂 My English is not very good, I don't know much about programming, and my analysis of things may not be accurate. But if this can help you, I will do my best. 🫡

Cloud2333 commented 6 months ago

New day, new bug. I forgot to test “Odin Sphere” and “Dragon's Crown” that night. 🫠 QQ截图20240315151627 QQ截图20240315152855 https://mega.nz/file/eMdBAaQb#8AzIQWL3lKT8UnOqcDpWPFkVtd2xhAKuIYVhL5e6o_k

Cloud2333 commented 6 months ago

Another discovery, .otm files in game files, appear to be appearing in vanillaware games for the first time, and appear to work with .mbs. QQ截图20240315191644 I think this might be files involved in changing colors, similar to fighting game .pat files. 示例 While working in conjunction with .mbs, the index of the texture area is established for color transformation. The file names and number of files also seem to match the number of variants in the game. Named characters have a special .otm . QQ截图20240315192209 I think this is a matter worth investigating, if there is any spare energy. 🤔

rufaswan commented 6 months ago

Interesting! That's why Galerius cape, the horse's hair are all white color in the QUAD PLAYER. They are meant to be blended with user selected color or pre-set colors for character customization.

Thanks for the discovery!

P.S. I accidentally mixed up ps4_sent (v76) and ps4_odin and ps4_drag (v72). It is fixed now.

Cloud2333 commented 6 months ago

Works perfectly now, thanks for the fix. 😽

Cloud2333 commented 6 months ago

I think my guess is correct, here are 2 characters with the same color. 11QQ截图20240316145247 These two characters have their own .otm files, but the content is exactly the same. QQ截图20240316145218 QQ截图20240316145229 Although I wanted to try to make a MOD and use other characters' .otm to a certain character, it failed. 🫠 I think I'll investigate for a while longer.

Cloud2333 commented 6 months ago

Tried removing the background and using the wrong awb, both worked, I will try to look into .otm 。 QQ截图20240316174502

Cloud2333 commented 6 months ago

Well, in the NS version of "Grim Grimoire OnceMore", the .otm files appeared, I think probably for testing, and even "Unicorn.mbs" appeared. 🤔 QQ截图20240316181139

rufaswan commented 6 months ago

Actually, Unicorn is an summon-able unit in Grim Grimoire. They have a lot of HP, low attack but high defense, it is like a tank.

rufaswan commented 6 months ago

Also, Grim Grimoire shares FTX with multiple MBS. For example, Charon.mbs, Fairy.mbs, Ghost.mbs and Morning_star.mbs are all using Flier.ftx. They are all flier unit type.

That's why an additional step with quad_prediv.php exists. I couldn't auto-detect them by filenames.

Cloud2333 commented 6 months ago

Hmm, interesting. I just tried to modify all the .otm, and the color of the character is still loaded normally. It seems that the problem is not in the .otm? And I just noticed that the .mbs used by the "Face_" series does not exist in the .otm. That one also has textures that need to be rendered in color. Where is .otm used? 🤔

Cloud2333 commented 6 months ago

https://news.denfaminicogamer.jp/interview/240308u/2#i-1 “I wrote a light gradation map shader, so I was able to change the color of each part by assigning numbers to them. In other words, parts that used to be drawn as one piece can now be divided into two parts and the colors can be changed.” It looks like this is similar to "rim light" and is done in real time in the game. And the "assigning numbers" actually remain a little bit on Galerius' texture.(BlackPrince_HG_M00.0) QQ截图20240316203559

Cloud2333 commented 6 months ago

I figured out the nature of the .otm file. It is an auxiliary script used to call audio files. In order to play the audio correctly with the appropriate action, work with .mbs . After I disabled all .otm files, the character's battle voices and sound effects disappeared, and the .otm file name actually matched the voice files. QQ截图20240316230358 So there is no need to investigate these files. 🫠

rufaswan commented 6 months ago

MBS also has sound effect ID in its animation data too, since PS2 Odin Sphere. It is played on the first frame when the character actually swing the weapon.

This reminds me Capcom vs SNK Pro animation data are also the same. The character's grunt voice HI-YA are random however, so they are not included/referenced in the animation data.

I guess OTM are random lines the character speak when certain animation is triggered?

Cloud2333 commented 6 months ago

Yes, OTM is a script that works with MBS to play voice and sound effects when executing an animation. Some animations will have multiple voices set. If the loading of the "Character" OTM file is blocked, the sound effects and voices will not be played in the battle. However, the voices in cutscenes are not affected by OTM, but the playing of character animation sound effects in cutscenes still relies on OTM. Although I would like to investigate further, this is beyond my capabilities. 🫠

Cloud2333 commented 6 months ago

I chose a relatively simple background, made a GB mod, and found a whole new level of fun. 😋 QQ截图20240317233228 QQ截图20240317233257

rufaswan commented 5 months ago

Thanks for your investigation on OTM files! Guess they really changed the sound effect ID to something else, or was replaced by new data entirely.

If you can disable certain file from loading, can't you keep track what files are actually used since booting the game? You can make a minimum / testing version of the game and mod that instead. You don't have to make a GB mod everytime for testing anymore :)

Cloud2333 commented 5 months ago

GB (Green Background) mod is just made for fun.😶‍🌫️ In fact, the way to make mods now is to mount an external cpk, which the emulator can easily do. As for "disabling from loading", it actually modifies the character header of the file or deletes the file content to make it invalid. Although it is a stupid method, it is really effective. 🫠

During the investigation, I noticed that there is a ".bbs" file in "BattleBG". QQ截图20240318200856 If you modify it so that it loads incorrectly, you will see an interesting picture. QQ截图20240318202557 QQ截图20240318204518 In addition to loading the complete BG together with other MBS, it also seems to have an impact on character image loading? The area with a special color is the setting area for rim light. And this file is quite sensitive. If too much content is modified, the game will crash directly, so it will take more time to confirm the true function.

Try changing .bbs to a special .bbs (Opening). The character is placed in a strange position. It seems to also control the information about the coordinates of the character's placement? QQ截图20240318235505

The ".asb" in "Script" is the core file that allows the game to run. I can't modify and investigate this. The ".fms" in "MsgSheet" is just a text file for the game. The ".inc" in "Debug" is probably the index text used by the game text. I have no clue about the ".drg" in "DbReg". I can modify it slightly to test whether it has any impact on the game.

rufaswan commented 5 months ago

OH! I thought you need to make a GB of .PPF or .XDELTA file for modding. BUT that is only for older console, newer console nowadays can be updated.

I was wondering how does rim light data even looks like. Good to know it actually on .BBS instead. Good work! :thumbsup: