educelab / volume-cartographer

Volumetric processing toolkit and C++ libraries for the recovery and restoration of damaged cultural materials
GNU General Public License v3.0
63 stars 21 forks source link

Implement volume-to-volume transforms #67

Closed csparker247 closed 11 months ago

csparker247 commented 11 months ago

This PR implements transforming meshes and PPMs between volume coordinate frames when using the various render applications.

Getting started

First, you need a transform JSON file (support added in #65) which maps from a source volume to a target volume. At the time of this writing, we only support affine transforms. Place transform JSON files in *.volpkg/transforms/:

{
  "type": "Transform3D", 
  "source": "20230205180739",
  "target": "20230905121646", 
  "transform-type": "AffineTransform", 
  "params": [[1.0, 0.0, 0.0, 0.0], 
             [0.0, 1.0, 0.0, 0.0], 
             [0.0, 0.0, 1.0, 0.0],
             [0.0, 0.0, 0.0, 1.0]]
}

The important fields are:

Applying the transform (automatic)

When working with segmentations, the transform is automatically applied when (1) you provide the --volume flag, (2) the provided volume is not the same as the segmentation's associated volume, and (3) there is a transform mapping from the associated volume to the target volume:

# maps from 20230205180739 -> 20230905121646
vc_render -v Foo.volpkg -s 20231121150000 --volume 20230905121646

For all invertible transform types (which includes affine transforms), we will also automatically invert the transform if the requested source and target volumes are the swapped equivalent of a transform's source and target.

# same transform as above inverted to map from 20230905121646 -> 20230205180739
vc_render -v Foo.volpkg -s 20231121150010 --volume 20230205180739

In the future, we plan to extend this automatic functionality to find a composite transform which maps between the source and target using multiple transforms.

Applying the transform (manual)

Of course, sometimes you need to manually specify a transform, either because the source volume can't be determined automatically or because you just want to have greater control over the transform being used. In this case, the --transform flag will let you select from a transform in the Volume Package or on disk:

# loads Foo.volpkg/transforms/my-transform.json
vc_render -v Foo.volpkg -s 20231121150000 --transform my-transform  

# loads ~/my-transform.json
vc_render -v Foo.volpkg --input-mesh bar.obj --transform ~/my-transform.json

Note: Specifying the transform manually does not set the target volume, but only transforms the 3D geometry. The default volume for the given input will still be used for rendering unless you also specify the --volume flag.

Controlling what gets transformed

By default, the transform is applied to the raw input mesh before any resampling or smoothing. You can change when the transform is applied using the --apply-transform-to flag. This flag has subtle but important effects on the output depending on the provided option:

Controlling the transform

A number of extra flags control the transform that is applied to the geometry:

App support

The following programs support transforms:

Other changes