PR-138: General Issues

[poisenbery]

I've been playing with the KKS version of the plugin from PR-138.

Notable issues:

Texture naming convention is different between the two games. KKS seems to short-hand texture names.

_MainTex is shorthanded to _MT _ColorMask is shorthanded to _CM _DetailMask is shorthanded to _DM _NormalMap and _NormalMapDetail are shorthanded to _NMP and _NMPD [new mexico police department?] _liquidmask is shorthanded to _lqdm _LineMask is shorthanded to _LM _ColorMask is shorthanded to _CM

This causes an issue during the "Import shader" step.

Normal maps in KKS seem to be a red color, instead of the black and white ones that usually get spat out in KK.

[MediaMoots]

This causes an issue during the "Import shader" step.

Could you share the card please?

[poisenbery]

This was actually my fault. I used the KKS version of the plugin without updating the blender scripts. Really sorry about this.

The only "issue" regarding textures is when the face type is 2, which is essentially the same as #163 So it's not really an issue then.

Again, really sorry and thank you for adding KKS support. I've been testing multiple cards and the improvements are VERY nice.

EDIT: Face3 does not successfully import. Now that Importing and Fixing are part of the same step, the solution in #163 isn't applicable.

Python: Traceback (most recent call last):
  File "C:\Users\austi\AppData\Roaming\Blender Foundation\Blender\3.0\scripts\addons\KK-Blender-Porter-Pack-master\importing\importbuttons.py", line 71, in execute
    bpy.ops.kkb.importeverything('INVOKE_DEFAULT'),
  File "D:\Steam\steamapps\common\Blender\3.0\scripts\modules\bpy\ops.py", line 130, in __call__
    ret = _op_call(self.idname_py(), C_dict, kw, C_exec, C_undo)
RuntimeError: Error: Traceback (most recent call last):
  File "C:\Users\austi\AppData\Roaming\Blender Foundation\Blender\3.0\scripts\addons\KK-Blender-Porter-Pack-master\importing\importeverything.py", line 910, in execute
    texture_error = get_and_load_textures(directory)
  File "C:\Users\austi\AppData\Roaming\Blender Foundation\Blender\3.0\scripts\addons\KK-Blender-Porter-Pack-master\importing\importeverything.py", line 504, in get_and_load_textures
    if not current_obj.material_slots['Template Face'].material.node_tree.nodes['Gentex'].node_tree.nodes['FaceMain'].image:
KeyError: 'bpy_prop_collection[key]: key "Template Face" not found'

location: <unknown location>:-1

How to reproduce: Export any character in KKS with Face type of 3. Attempt to import model using "Import Model" button.

[poisenbery]

The KKS version of the exporter is having issues with the Sailor type uniform when there is an accessory. [KKS console log]

[Info   :   Console] System.NullReferenceException: Object reference not set to an instance of an object

.new SMRData.ctor(SkinnedMeshRenderer)
void PmxBuilder.CreateMeshList()
void PmxBuilder.BuildStart_OG()

A folder with a few of the textures gets generated, but the PMX doesn't get built.

Jacket with accessory seems to be fine. Other types seem to function fine as well, but I haven't been able to test everything yet.

Edit: Some bras that grey out the underwear slot are having a similar issue where they will not export. I've included a sample card where it's present. .

[poisenbery]

Eyes are all opened way too much. I've been attempting to troubleshoot this for about an hour but am unable to identify the cause. From what I have observed, it is happening to every model. The effect is EXTREMELY obvious on characters with rectangle shaped eyes.

When I use a different method to import the PMX, the problem seems to still persist. I'm not sure if I have a mod conflict that is doing this. I made sure to disable anything affecting the eyes but I can't figure out why this one is happening. Adjusting the 'eye closed' shape key isn't able to get the eyes to the correct shape.

[MediaMoots]

I'm scared, mom plz save me...

JK

This is because KK sets the shapekeys to a certain value during runtime, but KKBP Exporter exports the original shapekeys from the mesh, so you can just set the shapekeys in blender to match the openness you want.

[poisenbery]

This is because KK sets the shapekeys to a certain value during runtime, but KKBP Exporter exports the original shapekeys from the mesh, so you can just set the shapekeys in blender to match the openness you want.

Sounds simple enough. I found that 0.1-.02 with the eye's closed shapekey seems to do the trick.

Was kind of concerning because it didn't happen in the KK version.

[MediaMoots]

The KKS version of the exporter is having issues with the Sailor type uniform when there is an accessory.

I think its fixed in 2.02, please try it out

[MediaMoots]

Was kind of concerning because it didn't happen in the KK version.

Oh really!

Could you share the card that has this issue in KKS but not in KK?

[poisenbery]

Could you share the card that has this issue in KKS but not in KK?

It's literally every single card I do with the KKBP exporter.

When I use the exporter that I made, it doesn't have the eyes opened like that. That's the main reason I'm so confused and think it's a bug.

I won't go into detail, but I'm extra familiar with the eyes on this particular character. .

[MediaMoots]

When I use the exporter that I made

Is it the one in this fork? https://github.com/poisenbery/KK-Pmx-Export--Redux

[poisenbery]

Is it the one in the fork?

https://github.com/Snittern/KoikatsuPmxExporterReverseEngineered

it's basically this with the KKS references. I haven't updated my fork with the compiled .dll yet because I haven't fixed the texture exporting yet.

The only thing this one does right now is export the PMX and some diffuse textures. I stopped working on it when I found the KKS plugins on here.

[MediaMoots]

Ok yeah, the old plugin had a bug where it exported the shapekeys with KK's corrections applied. but I fixed it in the new plugin so that it exports the original shapekeys.

So, it's not a bug.

[poisenbery]

Ahhhh I understand. It's expected behavior that I did not expect.

If only...I could read the source code ;)

[poisenbery]

I think its fixed in 2.02, please try it out

My dear Marnie is still having the same problem. From what I understand, it's her bra causing the issue. A few other of the "bras" that are a 1 piece set and grey out the underwear slot cause this as well. It's not all of them and it only seems to affect cards that I really like. I have a 2B card that causes the problem but it's too big to upload here.

The 2B card causing the issue can be found on Bepis by turning on "Featured Only" and hitting search. She'll be the 2nd card on the Koikatsu list.

Error Message:

[Warning:  HarmonyX] AccessTools.Field: Could not find field for type UnityEngine.GameObject and name objSleeves01
[Warning:  HarmonyX] AccessTools.Field: Could not find field for type UnityEngine.GameObject and name objSleeves02
[Warning:  HarmonyX] AccessTools.Field: Could not find field for type UnityEngine.GameObject and name objSleeves03
[Info   :   Console] System.NullReferenceException: Object reference not set to an instance of an object

.new ClothesData.ctor(string, int, ChaClothesComponent)
void PmxBuilder.CreateClothesData()
void PmxBuilder.BuildStart_OG()

[MediaMoots]

My dear Marnie is still having the same problem.

Thanks for all the testing!

I think its fixed now, please try out v2.04 in the PR

[poisenbery]

Sorry for the delay. I found a minor hiccup with the detail map texture. It was being exported but wasn't referenced at all in the shader pack or blender script itself.

I added a new texture node and named it "BodyNormD"

And then added this line of code to the "importeverything.py" script

Now they at least load in the textures group. I'm gonna update my stuff rn and see if the 3 Sleeves curse is gone. I was getting it on a LOT of outfits.

Thanks for all the testing!

You saved me SO much time by adding KKS support so the thanks really goes to you. Sorry for initially doubting you.

[poisenbery]

3 Sleeves curse is gone. The console spits out warnings instead of stopping now. Yay!

[FlailingFog]

but wasn't referenced at all in the shader pack or blender script itself.

To be honest, I'm not sure what that's used for...

[MediaMoots]

the detail map texture.

This adds detail to the normal map.

Check this out: https://github.com/xukmi/KKShadersRecreated/blob/e43980ee14c3a4b50dab2f0b45408125fb7b78d3/Shaders/Skin/KKNormals.cginc#L4

[poisenbery]

I'm not sure what that's used for...

Based on what I've found, all of the body line masks have an additional normal map. The line masks seem to have 2 parts: the color mask and the associated detail normal map [NMPD]. You already know how the color aspect works: it uses the line mask to simulate details on the body with different colors. This effect is further enhanced by the detail normal map.

Here's what it looks like. The line mask is applied in all cases: No normal maps:

Normal Map:

Normal Map + Detail Normal Map:

Here's the blue channel of the line mask. It's kind of like and inverse bump map:

And the alpha channel of it's associated detail normal map.:

[FlailingFog]

@MediaMoots So the BlendNormals function in Unity seems to perform a whiteout type of normal blending, and https://blog.selfshadow.com/publications/blending-in-detail/ says that a whiteout blend can be achieved with

float3 r = normalize(float3(normal1.xy + normal2.xy, normal1.z*normal2.z));

or

x and y = (normal1.xy + normal2.xy) / sqrt( 2 * (normal1.xy + normal2.xy) ^2 + 1) 
z       = 1/sqrt( 2 * (normal1.xy + normal2.xy) ^2 + 1)

I also see from https://github.com/Perfare/AssetStudio/issues/593 that the normal map alpha can be used to fill in the X and Y channels of the normal, with the Z channel as 1, so the BlendNormals can be replicated in shader nodes with this...

But that produces a gray normal map like the image below. I'm expecting a purple normal map, so I feel like I'm not using the texture correctly.

Dividing the normal1.xy and normal2.xy by a factor of two produces something that looks correct though. Do you know if using the just the alpha channel by itself is the correct way to reproduce the normal map, or do you have to do something else to it?

@poisenbery How are you applying the second normal map?

[poisenbery]

How are you applying the second normal map?

float3 n1 = tex2D(texBase,   uv).xyz*2 - 1;
float3 n2 = tex2D(texDetail, uv).xyz*2 - 1;

float3x3 nBasis = float3x3(
    float3(n1.z, n1.y, -n1.x), // +90 degree rotation around y axis
    float3(n1.x, n1.z, -n1.y), // -90 degree rotation around x axis
    float3(n1.x, n1.y,  n1.z));

float3 r = normalize(n2.x*nBasis[0] + n2.y*nBasis[1] + n2.z*nBasis[2]);
return r*0.5 + 0.5;

From: https://blog.selfshadow.com/publications/blending-in-detail/

There are....a few ways to do it. Just don't use a mix RGB node lol node.zip I THINK I did it right

[MediaMoots]

Thats pretty good, here is the source code for that function https://github.com/TwoTailsGames/Unity-Built-in-Shaders/blob/6a63f93bc1f20ce6cd47f981c7494e8328915621/CGIncludes/UnityStandardUtils.cginc#L144

[FlailingFog]

Thanks for the extra info. I put both methods into the "Raw shading" group

There are....a few ways to do it. Just don't use a mix RGB node lol

I originally had a toggle for swapping between the whiteout normals and your node group, but both the mixRGB and the mix shader nodes weren't working, so you may be on to something...

[poisenbery]

But that produces a gray normal map like the image below. I'm expecting a purple normal map, so I feel like I'm not using the texture correctly.

Node behavior is inconsistent when you use grey slots for vector information.

This can be proven with this setup:

The vector [1,0,0]*1 should result in the same vector, instead you get [1,1,1]. This is because grey slots ignore everything that is not the first component.

Grey sockets are 8bit[?? i think. not my specialty] float values. Purple sockets are 3D vector values [X,Y,Z] Yellow sockets are 4D vector values [R,G,B,A] And green is not applicable in this situation because it is shader data.

So, if you're doing math with vector information, the only time you can use the grey sockets is after a separate XYZ or RGB node. If you're adding vector information, you HAVE to use either a purple or yellow socket

float3 r = normalize(float3(normal1.xy + normal2.xy, normal1.z*normal2.z));

x and y = (normal1.xy + normal2.xy) / sqrt( 2 * (normal1.xy + normal2.xy) ^2 + 1) 
z       = 1/sqrt( 2 * (normal1.xy + normal2.xy) ^2 + 1)

Pretty sure these are correct

[poisenbery]

"Export Variations" does not check for additional outfit slots.

Any character in KK that has an additional outfit slot will not have that outfit exported. In KKS, the default outfit slots are 4, so any KK card used in KKS will only have the first 4 outfits exported.

The additional outfit slots in KKS seem to cause a game-breaking bug with the "Breast Physics Controller" mod. Console goes insane, multiple outfits can be selected at once, and the options to change the outfits disappears entirely .

Doesn't happen if you disable the controller for that card. Should be done BEFORE swapping outfits. .

[poisenbery]

"Export Variations" does not check for additional outfit slots.

Any character in KK that has an additional outfit slot will not have that outfit exported. In KKS, the default outfit slots are 4, so any KK card used in KKS will only have the first 4 outfits exported.

@MediaMoots Is it still recommended to not use the current version of the PR? I found some other bugs, but then I noticed this, so I didn't post anything about them.: https://github.com/FlailingFog/KK-Blender-Porter-Pack/pull/138#issuecomment-1161338393

[MediaMoots]

@poisenbery You are going to need to ask @FlailingFog about that because I haven't tested the Blender Import pipeline.

Any character in KK that has an additional outfit slot will not have that outfit exported.

Fixed in KK and probably fixed in KKS: V3.66 ad1e80c

[FlailingFog]

The default options for the blender plugin should work now. The "Rigify armature" option and "Pause to categorize" option should also work.

[poisenbery]

Fixed in KK and probably fixed in KKS

@MediaMoots Confirmed fixed! Yay. But, it now will ONLY export all outfits. It also doesn't seem to play nicely with the Pushup mod. Pushup changes the chest shape per outfit, but it seems like export variations only exports one body, so any card affected by pushup will have the breasts misaligned with the top itself. For bathing suits and other types of revealing clothing, the alpha mask cannot fix the problem.

One other thing I noticed is that multiple exports seem to take an extremely long time. It gets worse the more you do, and exiting the character creator doesn't seem to fix it. Exiting the game does. I haven't tested in KK but it is there in KKS [at least mine].

@FlailingFog I checked the import process and I didn't notice any problems.

[MediaMoots]

But, it now will ONLY export all outfits.

it's all or nuthn' ma man! Jk, maybe I'll add an option to only export the current outfit, lol.

It also doesn't seem to play nicely with the Pushup mod.

Impressed you found this issue so quickly; It took me a bit to figure that one out. I think the only way it can be fixed is to just disable pushups during the export, I'll deal with that in a bit. (Working on some other features now)

One other thing I noticed is that multiple exports seem to take an extremely long time

Ok, thanks for the feedback.

[MediaMoots]

@poisenbery

Check out 72b9efb : )

[poisenbery]

Impressed you found this issue so quickly; It took me a bit to figure that one out.

https://github.com/FlailingFog/KK-Blender-Porter-Pack/issues/84#issuecomment-1019560727

I am actually psychic.

[poisenbery]

Check out 72b9efb

It still refuses to do single outfit exports in KKS =/

[MediaMoots]

https://github.com/FlailingFog/KK-Blender-Porter-Pack/issues/84#issuecomment-1019560727

I am actually psychic.

It still refuses to do single outfit exports in KKS =/

It can't refuse what it wasn't made to do; Soon tho, soon.

[poisenbery]

It can't refuse what it wasn't made to do;

ahhhh my bad. the overload of anime girls seems to be having a negative effect on my reading comprehension