OBJExporter (also PLY and STL): Support for deformable meshes (skins & morphs)

[willstott101]

Is your feature request related to a problem? Please describe.

We have a simulation using Three.js making use of both skinned meshes and morph targets. We want to be able to export a static mesh at a given frame. The OBJExporter in some ways works very well for this, however it doesn't deform vertex positions when saving.

Describe the solution you'd like

I'd like a way for the OBJExporter to be able to access a deformed version of meshes. This could also be used by the PLYExporter and STLExporter.

Describe alternatives you've considered

Exporting to GLTF and using a different tool to translate to OBJ - this is quite a janky workflow for me.

[donmccurdy]

I could see this being a good addition to BufferGeometryUtils, adding a function that takes a snapshot of a Mesh or SkinnedMesh, and returns a new BufferGeometry with morphs and skinning applied. The existing computeMorphedAttributes utility will get you halfway there.

Depending on how complex that code is (computeMorphedAttributes is currently not trivial...) we could possibly invoke it in exporters, but I suspect that might need to be a bit of userland pre-processing.

/cc @gkjohnson possibly relevant to your pathtracing work?

[willstott101]

Ahah, that function does look helpful indeed. Why does the code complexity make you think it ought be a user-level pre-process step? The time taken to export?

Requiring a user to perform this as a pre-process step sounds kinda heavy - both in terms of resources and the possibility for users to invoke it incorrectly. I have a scene that exports to a 30MiB OBJ - having to clone that entire scene in order to modify all of the contained mesh data - to use once as I chuck it in to an exporter and then subsequently throw it away seems very wrong to me. The exporter can perform these on a mesh-by-mesh basis, minimizing RAM use.

I suppose there's an argument that if you were doing it every frame you would want to keep the morphed attribute arrays around and re-use the buffers, but the exporter invoking computeMorphedAttributes wouldn't preclude a user making an optimisation like that anyway.

The GLTFExporter currently takes a whole bunch of options, OBJExporter could perhaps take an options parameter too, and I should think it'd be possible to figure out in the exporters if it needs doing anyway requiresMorphedAttributes for instance?

[gkjohnson]

@donmccurdy

/cc @gkjohnson possibly relevant to your pathtracing work?

The StaticGeometryGenerator class recently added to three-mesh-bvh (docs here) will generate a new static geometry / mesh based on one or more passed objects with and without deformable geometry. It's designed to update a provided geometry in place to avoid the memory and performance overhead of creating a new object every run. It was originally made to enable the path tracing / BVH libraries to support skinned geometry and morph targets. It supports any morph target / skinned geometry attributes and transforms normals and tangents correctly, as well.

I anticipate maintaining it in that project so I can control performance etc of the function. Maybe some version of it could be added in this project? Though I think it makes more sense to just promote the use of one.

@willstott101

Requiring a user to perform this as a pre-process step sounds kinda heavy - both in terms of resources and the possibility for users to invoke it incorrectly. I have a scene that exports to a 30MiB OBJ - having to clone that entire scene in order to modify all of the contained mesh data - to use once as I chuck it in to an exporter and then subsequently throw it away seems very wrong to me.

Any computation required to happen in userland will have to happen in the exporter, as well. In terms of memory overhead - have you run into any issues? If not then this seems like a pre-emptive optimization. I'm sure everyone would like to keep the code in the exporters as simple as possible considering how many of them there are to maintain. An external processing approach is more flexible and will let every exporter just work.

[donmccurdy]

I anticipate maintaining it in that project so I can control performance etc of the function. Maybe some version of it could be added in this project? Though I think it makes more sense to just promote the use of one.

I'll defer to you here, if it's easier to maintain in a separate repository that is fine!

The performance costs of (1) copying and mutating vertex data and (2) serializing it all to ASCII strings for OBJ or PLY, are going to greatly outweigh the cost of shallow-cloning the wrapping objects. And adding new dependencies to exporters complicates their distribution as well as maintenance — e.g. consumers from examples/js/exporters need to import additional files manually. It would be nice to have a short snippet showing how to export a particular frame as a static mesh but I'd lean away from building the option into various exporters directly.

[willstott101]

Hm, well I can't clone my scene cause my userData isn't JSON serializable (maybe I'll make a PR to use structuredClone). I've just copied the OBJExporter into my source tree and modified it for now :shrug:

Looks like computeMorphedAttributes has worked nicely for my skinned meshes, but not for morph targets so I'll dig into that before trying to close this issue with a helpful example to the next passerby

[willstott101]

Ok well my final workaround (to avoid .clone()) is as such:

import { OBJExporter } from "three/examples/jsm/exporters/OBJExporter";
import { computeMorphedAttributes } from 'three/examples/jsm/utils/BufferGeometryUtils';

function withMorphedAttributes(root, callback) {
  const attrs = [];
  root.traverse((o) => {
    if (o.isMesh || o.isLine || o.isPoints) {
      const morphed = computeMorphedAttributes(o);
      attrs.push(morphed);
      o.geometry.setAttribute("position", morphed.morphedPositionAttribute);
      if (o.isMesh)
        o.geometry.setAttribute("normal", morphed.morphedNormalAttribute);
    }
  });
  try {
    callback(root);
  } finally {
    let i = 0;
    root.traverse((o) => {
      if (o.isMesh || o.isLine || o.isPoints) {
        const morphed = attrs[i];
        o.geometry.setAttribute("position", morphed.positionAttribute);
        if (o.isMesh)
          o.geometry.setAttribute("normal", morphed.normalAttribute);
        i++;
      }
    });
  }
}

// How to use when exporting
withMorphedAttributes(world.scene, (root) => {
  const data = new OBJExporter().parse( root );
  download("Scene.obj", data);
});

Thanks for your pointers. As far as I'm concerned this can be closed now, I'm not sure an object-returning specialised version of clone would be massively helpful over what I have above. I didn't realise BufferGeometryUtils was an example originally - and I agree that using it from exporters ain't a pretty idea. I don't think computeMorphedAttributes would be out of place on the BufferGeometry class in core personally, which then I might argue exporters should call it. But honestly I don't think it's a big deal.