Art: Scillia VRM finalisations

[Vianvolaeus]

We need to do some testing, optimization and mastering of the final VRM for the #1382 character.

Art:

• [x] replace shoes with tech shoes
• [x] bring color (saturation) in line with other default avatars in app

Technicals:

• [x] Blendshapes under VRoid naming convention (Expressions, AEIOU, etc)
• [x] test hair bone naming convention in webaverse
• [x] reduce filesize
• [x] reduce hair transforms
• [x] rigging / anims correct in webaverse

[Vianvolaeus]

We can, as a (destructive!) edit once everything is absolutely final on the materials, bake everything out into a texture atlas / bake in Blender. This does limit per material in-engine editing, of course, but will reduce drawcalls & texture usage.

Even if it's not used on the main characters, it's something to consider for 'static' NPCs, etc.

[avaer]

It's totes fine to do that destructive edit/mixdown, but is there any way to make it automatic vs manual? So we can do it again later?

[Vianvolaeus]

Right now the flow would be final master file from Blender > bake down materials via script/plugin in Blender > export to Unity > remove unused shapekeys on VRM export via uniVRM.

Not sure what you mean by do it again later. On the same model, or for others?

[Vianvolaeus]

Unity had been upscaling textures to 2k automatically, so that's another tripwire to watch out for. Should substantially reduce filesize, especially when combined with the blendshape export settings. Seeing a large reduction already. (Full res, no compression, all blendshapes is about 140MB - correct sizing, compression, and removing blendshapes not referenced by BlendShapeClips in VRM is about 30MB)

[avaer]

Not sure what you mean by do it again later. On the same model, or for others?

On the same model, after edits. I don't see any more needed now (other than the above) but it's likely to be needed in the future.

[avaer]

30MB is still a little bit large; any way to halve that?

[Vianvolaeus]

Even after very aggressive texture compression, we're not seeing much in the way of filesize reduction. If I remember correctly, gltf is quite bloated in regards to shapekey filesize compared to other formats. This is with everything nuked except blink / AEIOU and ALL.expressions (Joy, etc)

We'll have to implement the above material work, but even with a singular blank mat across the board exports are still hitting 10MB+.

As for rebaking on the same model, it depends on the type of edit. If it's a texture edit, it can be rebaked and the material can be updated on the VRM. If it's a geometry edit then you're going to break the model anyway, so you'd have to redo the process.

If it's something like a clothing swap, then we should be able to have a baked material base model, and then bake out the clothing set as it's own material.

This is going to take more time to finish off than I'd hoped. Unity has decided gravity is backwards for spring bones, which needs debugging.

[Vianvolaeus]

Only other thing I can think of is a misnamed UVmap that's adding an extra to the count (VRoid content uses a different naming scheme for whatever reason, which has to be manually changed) - doesn't look like that's the case, since the model I was using is merged and only has a single map.

[Vianvolaeus]

Trying to test in-engine and it seems like weba is constantly refreshing, along with some other issues. Filing now.

1484 , #1485 , #1486

the TL;DR here is that I can't login via Discord account to mint on the FE (where is the minting frontend now, too?), the bot isn't working as a workaround, and even when I'm in the app, it seems to be restarting constantly.

[avaer]

Thanks for filing! Definitely bugs. Looking into it.

[Vianvolaeus]

Atlassing and reducing material count does help on the filesize front. Unity seems either incorrectly read or otherwise change externally compressed textures... will have to look into this. Compressing internally doesn't appear to export correctly. Unity thing. Either way, atlassed mats with Diffuse / Normal only for MToon, as well as reduction of geo normals for morphs, etc, appears to put us under the 15MB target.

I still need to test within Weba for naming conventions and animations / rigging, but having problems @ mint

[avaer]

atlassed mats with Diffuse / Normal only for MToon, as well as reduction of geo normals for morphs, etc, appears to put us under the 15MB target.

That sounds like the right solution, thank you researching this.

I still need to test within Weba for naming conventions and animations / rigging, but having problems @ mint

Yup, bugfixing that.

[Vianvolaeus]

As part of this process and our goal to not rely on Unity, I've been extensively using the VRM exporter plugin. The recent discovery of the experimental UI looked promising, but is currently fraught with multiple issues (on top of being a a clunky experience anyway)

I'll be running it through Unity for the time being.

Problems with it are as follows:

Using UI to set bones in hierarchy appears to not export correctly - Bone specified in the UI does not update the associated files - for example, if you change or assign a bone for left shoulder, it will not update the associated line(s) in json files / not associate that bone with humanoid definition on export.

Can't create new blendshape references in UI - Cannot assign a blendshape to a VRMBlendShapeClip. This is relevant to the BlendShapeProxy script in VRM in which a shapekey is used to define an emotion or viseme. For example - the VRM application may/will look for A/E/I/O/U on the VRM, which may have been created from a shapekey for 'mouth open', etc.

I could not get this to work manually via script, but it may need more testing.

Can't create new spring bone references in UI - Cannot define a bone as being a spring bone / physics bone. Oddly enough this does appear to work if set manually via scripting, manually adding it into the springbones.json file. Very very time consuming though.

Export menu suddenly ramps up cpu usage - Never seen this behaviour in an exporter menu - actual export process can be perf. intensive, but just the menu itself is extremely bad, maxxing out a 5950x. Something isn't right here.

Export of VRM can crash Blender - this is odd and inconvenient behaviour. Interestingly Blender did not generate an autosave on crash, which it usually does.

Filesize bloat for unknown reason (I suspect this is animation / shapekey related, as is an outstanding problem with gltf - but there are no controls for it on export)

VRM validity checks incorrect - exporter will report validity is OK, but will export with incorrect data in jsons, meaning that the VRM will not work correctly, since it can reference incorrect bone references, etc.

Saving json files does not update the json that is exported, even if the VRM exporter reports validation - I theorised that the json files were not exporting correctly if manual changes were not being saved. A export/reimport shows that they simply do not export correctly.

exported VRM, even with upper chest present, does not equip in Webaverse - likely a combination of above issues - while there may be an armature in the model itself, if the vrm meta isn't correct, then the app will not be able to read it.

[avaer]

That's rough, thanks for the deep research. It seems like this plugin could be improved to not be broken in the experimental mode, but if it's this full of holes we might need to stick to Unity.

Btw can you list the plugins that have been tried for posterity?

[Vianvolaeus]

The above is referring to https://github.com/saturday06/VRM_Addon_for_Blender.

A couple of the issues I had have been filed already with no response.

Attempted to get https://github.com/kitsune-ONE-team/KITSUNETSUKI-Asset-Tools working a while ago to no avail, but it appears to have been updated in the past month or so, so I can revisit it.

Unity is the best solution for now.

[Vianvolaeus]

Unity & uniVRM, while a bit more friendly on the UI side, also has it's own pitfalls.

Unity will occasionally appear to not import an FBX correctly with modifiers enabled. While this is isn't a limiting problem in terms of functionality, it does mean that changes have to be applied prior to export (destructive) if we don't want to gamble on Unity working or not working.

The problem I was having here was in regards to shapekeys not being present on the export with the Apply Modifiers setting enabled in the Blender exporter - yet just one export prior, it had worked completely fine (no edits, just wanted a clean import in a new project)

uniVRM tries to extract all textures from the model according to the unity console, even though they may not be used in the current shader setup, which will halt / abort the VRM exporting process. (In this case an unused roughness map halted the process - shaders were changed to gtlf unlit)

Lastly - springbone component code appears to rely on having a child bone present (even if unweighted). This is a bit of a pain for interop to other formats since it adds bone count in other physics solutions. There is an option to add leaf bones (empty child bones) on the initial export of a model from Blender which would be an 'autofix', at the cost of a additional +1 child bone on every bone chain of a model.

[Vianvolaeus]

Scillia V1 is in the drive! https://drive.google.com/drive/folders/1S_qIKD_0OUmhymwsD55ZiYLxC5k1GaPy

[Vianvolaeus]

Comments from @avaer on v1:

"for scillia.vrm I think we gotta age the proportions up a little to get it out of child zone, test a scar, and maybe improve the hair folds a bit but it's good enough to at least be able to see how it fits with a set of other models"

[Vianvolaeus]

I pushed a minor asset update to the drive this morning: https://drive.google.com/file/d/1hz3n_JwwCl8YHprFmg21gaSokR6PZZCH/view?usp=sharing

It is simply a shader update - it should now be shadow reactive thanks to the new toon shader implementation / lighting updates.