Fix bone weights and vertex normals for secondary geometry

[laura-a-n-n]

This PR addresses issue https://github.com/laura-a-n-n/silent-hill-museum/issues/2 and uncovers some mystery about "extra" geometry (which is usually hair).

In particular, it locates where the bone weights and bone indices are located in each "extra" vertex.

I've also resolved to leave in the computation of normals for now. There does seem to be a normal vector in every vertex for secondary geometry, but using these normals seems to give inconsistent results when importing into Blender, so I'm leaving them out until I decide what to do with them. I could also add this in as an option to the GUI, but I'm not sure how I feel about that. If you have opinions feel free to let me know.

Also, an important caveat for making the model as seamless as possible is to turn on Alpha Clip for ALL materials, not just the first one in each mesh. (I messed this up at first...)

I'm attaching the resulting .blend file (as a .txt) from this patch. @Raq1 please rename to .blend and confirm that the hair looks better to you?

hhh_mar.txt

[Raq1]

Yep, I can confirm that the hair weights look correct in the blend file you sent!

By the way, I have barely seen anyone mention this, probably because the project wasn't really "advertised" around so very few people know about it, but around the start of the year someone managed to make a Noesis importer/exporter for SH3's .mdl format, allowing not only to extract models but also to replace the assets by using Noesis' export function. Normal vectors are included.

I'm aware that SH3's .mdl format is a bit different, but perhaps the specific normal format may be unchanged? Maybe a look at the code may help you find out the format.

The plugin also imports morph/blendshape data (however you want to call it) for facial animations, maybe that can be of your interest too.

Here's the repo if you want to look at it: https://github.com/alanm20/SH3_chr

[laura-a-n-n]

oh my god no way haha. taking a cursory look at this, it has a lot of the same logic that i went through hell and back to reverse engineer for silent hill 2... if i had known about this it would have been really helpful! that said, i tend to be a little stubborn about puzzles, so in a way i'm glad the answer wasn't "spoiled" for me :) anyway that's so badass, this is gonna be super helpful for getting silent hill 3 mdls working in the museum :)) thanks for sharing and reviewing the blend file!

[Raq1]

For sure, it's always fun to also reverse and find out things for yourself :P I'm glad it could be useful!

[laura-a-n-n]

somehow this fix didnt make it into latest deployment? sorry about that, it's live now :)

[Raq1]

Oh yeah, I've been wondering :p by the way, in case you didn't know about this either, someone made a Blender plugin to import SH2/SH3 PS2 models. I'm not sure how different the format is between the two platforms but it may be worth taking a look at their normals code perhaps: https://github.com/Murugo/Misc-Game-Research/releases

[laura-a-n-n]

yeah sorry again about any inconvenience xD

i have seen that project! murugo is a contributor to sh2:ee. unfortunately, i've never got that blender plugin to work or even load for me, which was one of many reasons i started this project. i tried it a couple months ago before i knew anything about how all this worked, or anyone in this community!

also, having taken a closer look, their normals code doesn't seem to apply to the pc version, because it unpacks four int32s for each vertex (being the x, y, z, w of each normal), but there are only 8 unknown bytes in each vertex of the "main" geometry, which would only leave room for two int32s. also, the vertex positions in the pc version are float32s, not int32s... so, i still suspect that the "main" geometry doesn't specify normals, and they're just computed at runtime. that being said, it is strange that the "extra" geometry does seem to specify normals.

(unrelated, but we just discovered that the "extra" geometry is there, at least in part, for transparency! turns out "main" geometry seems to only support 1-bit alpha and specularity, but "extra" geometry can support proper transparencies)

[Raq1]

Yeah I was actually imagining that extra meshes were being used for transparency, given they seem to always be used for hair for character models.

Hmm, if the PS2 format specifies normals, I'm pretty sure the PC one does too though. Maybe the normals are hidden somewhere in those 8 bytes, maybe they're not regular floats or ints but they're doing some specific algorithm to store them. Maybe something like 16 bits per normal?

x = 16bits y = 16bits z = 16bits w = 16bits

Old games do all kinds of weird things to store vertex components to preserve size, mine is just a wild guess but those 8 bytes definitely must be normals, otherwise I'm not sure what else they could be.

[laura-a-n-n]

@Raq1 i'd say dig into it yourself! visit https://ide.kaitai.io/, paste in mdl.ksy, and then drag & drop any mdl file in. now if you use the bottom panel, you can find the vertex list and look at the "unknown" section inside each vertex.

i've done a lot of thinking about this, and i can't find any way that they would (at least directly) encode the normals. when you interpret them as 16-bit floats or ints, they explode to unreasonably large or extremely small values.

if you want an avenue of research, it's possible that these values somehow relate to the cluster nodes, which murugo's code seems to refer to as morph data, but i really haven't looked into it so i dunno. i might ask around in the community about that section

[laura-a-n-n]

ohh, i just got some interesting intel about these fields! apparently it's possible that these are quaternion values, computed in a funny way! but word is that this relates to animation data somehow?

edit: nvm, that was outdated information xD from an REing attempt many years ago

[laura-a-n-n]

@Raq1 great news! i spoke with murugo and you were right about the normals :) i found it very weird that they are int16s (especially because the transparent groups store them as float32s), but they are there!!! thank you for pushing me to ask about it :D

see the commit here, and feel free to test it out & let me know your thoughts, it's live right now

(oh btw, the code in murugo's repo atm isn't compatible with the pc format, so what i said above holds; but he sent me a copy that does support pc stuff!)

edit: there's still something weird about the transparency in maria's hair whenever i import to blender, but i don't get this in the museum >_> hopefully i'll figure this out soon

[Raq1]

Hmm, I'm not sure this looks quite right :p

edit: there's still something weird about the transparency in maria's hair whenever i import to blender, but i don't get this in the museum >_> hopefully i'll figure this out soon

If you want to look at the transparency in Blender in the most accurate way possible, the best way is to enable Alpha Hashed. It's basically using the same method of rendering as modern videogames, shows dithered alpha but doesn't have any issues with rendering overlaying strands of hair with transparency.

[laura-a-n-n]

xD lol whoops i broke it

there's a little bit of processing needed to go from int16s to float32 normals, i'll fix it soon sorry

i reverted the changes from the previous version btw, so we're back to computing normals for now; it was late last night and i made some mistake :/