Original MmdMesh Object Fails isMmdSkinnedMesh Check After Creating MmdModel

[SavAct]

After creating an MmdModel using this._mmdRuntime.createMmdModel(), the original mmdMesh object no longer passes the MmdMesh.isMmdSkinnedMesh() check. This behavior seems unintentional and affects the BpmxConverter:

console.log("is MMD mesh", MmdMesh.isMmdSkinnedMesh(mmdMesh));  // true

mmdModel = mmdRuntime.createMmdModel(mmdMesh);

console.log("is MMD mesh", MmdMesh.isMmdSkinnedMesh(mmdMesh));  // false

new BpmxConverter().convert(mmdMesh);  // Error: {model name} is not MmdMesh

[noname0310]

That is the intended behavior: after running createMmdModel, the mmd metadata is removed from the mesh to save memory. Also, for a second reason, the mmd runtime changes several values in the model at runtime. Because of this side effect, using a model controlled by the mmd runtime in a bpmx converter may result in unexpected behavior.

But there is a way to do it if you want to You can recover the metadata if you store the metadata somewhere else before running createMmdModel.

You can use this test code as a reference https://github.com/noname0310/babylon-mmd/blob/main/src/Test/Scene/wasmMemoryTestScene.ts#L82-L102

[SavAct]

When I try that, the operation succeeds, but the resulting file is significantly smaller, likely due to missing textures, as shown in the picture:

[noname0310]

Sorry, there were a few things I didn't say.

The default option for the texture loader is to delete the buffer after loading, this is for memory savings but should be turned off when using converters. You can turn it off with the following code

const pmxLoader = SceneLoader.GetPluginForExtension(".pmx") as PmxLoader;
const materialBuilder = pmxLoader.materialBuilder as MmdStandardMaterialBuilder;
materialBuilder.deleteTextureBufferAfterLoad = false;

Also, the pmx loader does not load parameters into babylon.js that are not used by babylon-mmd by default. If you want to convert all data from pmx losslessly, use the following code

pmxLoader.preserveSerializationData = true;

You also need to check the conversion options in BpmxConverter. For example, if the mesh you want to convert is a stage and not a humanoid, you may not need skinning data or morph data.

const arrayBuffer = bpmxConverter.convert(mesh, {
    includeSkinningData: false,
    includeMorphData: false
});

And the translucentMaterials and alphaEvaluateResults options are data that indicate how the mesh should be shaded.

If these are not filled in, a specific algorithm will be used at load time to determine whether the mesh is opaque or translucent.

/**
 * BPMX convert options
 */
export interface BpmxConvertOptions {
    /**
     * Include skinning data into BPMX data (default: true)
     */
    includeSkinningData?: boolean;

    /**
     * Include morph data into BPMX data (default: true)
     */
    includeMorphData?: boolean;

    /**
     * Array that stores weather the material is rendered as translucent in order of mmd materials metadata (default: [])
     */
    translucentMaterials?: boolean[];

    /**
     * Array that stores material alpha evaluation result in order of mmd materials metadata (default: [])
     */
    alphaEvaluateResults?: number[];
}

Determining whether a material is opaque or translucent is impossible for an algorithm to do perfectly, so pmx converter provides a Fix material feature that allows you to manually specify how the material should be rendered using the GUI.

Fully automating it may cause the rendering results to look strange. However, if you want to perform a fully automated conversion, you might want to write your code like this

const textureAlphaChecker = new TextureAlphaChecker(scene);
const pmxLoader = SceneLoader.GetPluginForExtension(".pmx") as PmxLoader;
pmxLoader.loggingEnabled = true;
pmxLoader.preserveSerializationData = true;
const materialBuilder = pmxLoader.materialBuilder as MmdStandardMaterialBuilder;
materialBuilder.deleteTextureBufferAfterLoad = false;
// Load as alpha evaluation to let the pmx loader determine the alphaEvaluateResults result.
materialBuilder.renderMethod = MmdStandardMaterialRenderMethod.AlphaEvaluation; 

const mmdMesh = await SceneLoader.ImportMeshAsync(
    "", "your/model/path/", "model.pmx", scene
).then(result => result.meshes[0] as MmdMesh);

const translucentMaterials: boolean[] = new Array(materials.length).fill(false);
const alphaEvaluateResults: number[] = new Array(materials.length).fill(-1);

const meshes = mmdMesh.metadata.meshes;
const materials = mmdMesh.metadata.materials;
for (let i = 0; i < materials.length; ++i) {
    const material = materials[i] as MmdStandardMaterial;
    const diffuseTexture = material.diffuseTexture;

    if (material.alpha < 1) {
        translucentMaterials[i] = true;
    } else if (!diffuseTexture) {
        translucentMaterials[i] = false;
    } else {
        translucentMaterials[i] = true;
        const referencedMeshes = meshes.filter(m => m.material === material);
        for (const referencedMesh of referencedMeshes) {
            const isOpaque = await textureAlphaChecker.hasFragmentsOnlyOpaqueOnGeometry(diffuseTexture, referencedMesh, null);
            if (isOpaque) {
                translucentMaterials[i] = false;
                break;
            }
        }
    }

    alphaEvaluateResults[i] = material.transparencyMode ?? -1; // just fill with evaluated results
}

const arrayBuffer = bpmxConverter.convert(mesh, {
    includeSkinningData: true,
    includeMorphData: true,
    translucentMaterials: translucentMaterials,
    alphaEvaluateResults: alphaEvaluateResults
});

There's no documentation to explain this yet, so please feel free to ask questions if you don't make sense.