Avatars shader coalesce

[avaer]

Most avatars used have duplicate materials (20-50) and they are terribly optimized, since the author is assuming that the game engine will fix the inefficiencies. This is indeed the case usually so it works.

We already do this in Webaverse for avatar crunching, but it seems we can get rid of 80-90% of the avatar materials even in high quality mode, due to this massive redundancy.

For example, a typical avatar of ours only has 3 distinct materials despite having 25 materials. At a first approximation, crunching these would result in an 80% reduction in avatar render time with no render cost (though we would need to crunch this avatar in offscreen engine before wearing it, which seems reasonable to me).

https://github.com/webaverse/app/blob/master/avatar-cruncher.js

[avaer]

Note that this technique can be easily applied to pets, GLBs.

[avaer]

ms = [];
os = [];
vrmApp.traverse(o => {
    if (o.isMesh) {
        m = o.material[0] ?? o.material;
        if (m.visible !== false) {
            ms.push(m);
            os.push(o);
        }
    }
});
/* is = ms.map(m => m.map?.image).filter(i => !!i); is.forEach(i => document.body.appendChild(i)); */

set = new Set();
for (let i = 0; i < os.length; i++) {
    k = renderer.getProgramCacheKey(rootScene, os[i], ms[i]);
    set.add(k);
}
console.log(set); // e.g. length 3

[avaer]

Can be added to THREE.js WebGLRenderer to visualize the cached programs:

this.getProgramCacheKey = (scene, object, material) => {
    const currentRenderState = renderStates.get( scene );
    currentRenderState.init();

    const lights = currentRenderState.state.lights;
    const shadowsArray = currentRenderState.state.shadowsArray;

    const parameters = programCache.getParameters( material, lights.state, shadowsArray, scene, object );
    const programCacheKey = programCache.getProgramCacheKey( parameters );
    return programCacheKey;
};