Tydra work items

[syoyo]

dev is a current active working branch for Tydra https://github.com/syoyo/tinyusdz/tree/dev/src/tydra , especially OpenGL/Vulkan friendly data structure conversion.

Tydra provdes a handy logic to convert some USD scene with material/texturing to OpenGL/Vulkan-like renderer. Limited support of keyframe animation, skinned mesh and blendshapes.

Phase1(mostly done)

Node hierarcy(scene graph), static mesh, material/textures

• Scene graph construction

  • [x] Xform(at a specific time)
  • [x] Mesh, GeomSubset

• Load asset from USDZ

  • [x] AssetResolution and FileSystem handler support for USDZ https://github.com/syoyo/tinyusdz/issues/140 https://github.com/syoyo/tinyusdz/issues/134

• Material/Texturing

  • [x] BindMaterial resolution(consider material bind strength).
  • [x] Basic UsdUVTexture/PrimvarReader support. UV primvar name resolution.
  • [x] Basic Texture image load(jpg, png, exr, ...)
  • [ ] Custom texture image loader? https://github.com/syoyo/tinyusdz/issues/143
  • [x] Linear-sRGB colorspace conversion of texture.

• Mesh conversion

  • [x] Convert mesh at a static time

• [x] Mesh Triangulation

  • [x] Triangulate common vertex attributes(normals, texcoords, vertex_colors, vertex_opacities)
  • [x] Triangulate GeomSubset indices
  • [x] Validate triangulation is correct

• [x] Vertex index buffer build

  • To render a mesh with single index buffer(OpenGL/Vulkan)
  • [x] Convert 'facevarying' primvars to 'vertex' primvars
  • [x] Validate index buffer build is correct

Phase2(mostly done)

• [x] Vertex Skinning

  • [x] Read vertex weights
  • [x] Build skel hierarchy(bone hierarchy)
  • [ ] Skinning transform matrix build(?)

• [ ] Animation

  • [ ] Node xform animation(timeSamples)
  • [x] SkelAnimation(rotation, translation, scale) https://github.com/syoyo/tinyusdz/tree/tydra-skelanimation

• [x] BlendShapes

  • [x] Basic BlendShape Prim support

• [x] Muli-value in Attribute support https://github.com/syoyo/tinyusdz/issues/154

• [x] Test with some USD files

=> merged into dev

Advanced(future task)

• AR object https://github.com/syoyo/tinyusdz/issues/163
• BlendShape
  • [ ] In-between blendshapes
• [ ] Optimize vertex buffer and vertex index build.
• Wide color texture support
  • [ ] Linear-displayP3 colorspace conversion of texture
  • [ ] ACEScg(Linear) colorspace conversion of texture.
• [ ] Subdivision surface
  • [ ] Subdivide surface using osd(OpenSubdiv)
  • [ ] Interpolate primvars
• [ ] Material collection
• [ ] MaterialX support
• [ ] Vertex animation of a mesh
• [ ] UDIM texture support
• [ ] Support some ACES color profile as a built-in
• [ ] Multiple Animation(SkelAnimation) support

[syoyo]

With https://github.com/syoyo/tinyusdz/commit/a1f1a06eac51039288efbb1b7549840f7f32b769 , at least single textured quad USC scene is now able to convert to RenderMesh/RenderScene.

tydra_to_renderscene models/uv-quad-textured.usda --dumpobj

Then open .obj with MeshLab

[syoyo]

Note on testing using tydra_to_renderscene example.

tydra_to_renderscene input.usd [--dumpobj] [--notri] [--noidxbuild]

--dumpobj Dump mesh as wavefront .obj
--notri Do not triangulate mesh
--noidxbuild Do not rebuild vertex indices(for raytraced renderer or mesh data conversion)

[syoyo]

DamegedHelmet.usdz can now be converted through

tydra_to_renderscene DamagedHelmet.usdz --dumpobj

then open .obj in MeshLab(or Blender)

[syoyo]

In Blender, set up-axis -Z to load .obj(exported from tydra_to_renderscene)

[syoyo]

Object capture example(No texture colorspace conversion is done yet though)

[syoyo]

To support animation correctly, we'll need to implement https://github.com/syoyo/tinyusdz/issues/154 since some USDZ file(e.g. robot_walk_idle.usdz has the following attribute

def SkelAnimation "Animation " {
  quatf[] rotations = ... # default value(value at `Default` timecode)
  quatf[] rotations.timeSamples = ...
}

as a workaround, rendermesh-refactor can support having default value and timeSamples(but uses timeSamples only when both are declared) in this commit: https://github.com/syoyo/tinyusdz/commit/bdedcc070b2acdf55c9ed0f041fa5f4a09db65d2

so at least SkelAnimation in some USDZ files can now be handled correctly(but no default time value though).

[syoyo]

tydra_to_rendermesh robot_walk_idle.usdz --dumpobj is now at least able to export meshes and skin weights . Next step is to convert Skeleton(skel hierarchy) and SkelAnimation(animation samples)

To support Skinning, I am working on RenderScene to USD exporter(read USD with Blender to test RenderScene's Skinning data is correctly converted)

[syoyo]

Initial success of supporting SkelAnimation convert in renderscene-refactor branch. (now in 'dev' branch)

https://github.com/syoyo/tinyusdz/assets/18676/e6e08770-c879-4419-b252-98118daf0937

Read USD -> Convert to RenderScene through Tydra -> Export USD again(tydra_to_renderscene --dumpusd) -> Read it in Blender 4.1

[syoyo]

Initial support of BlendShape read/export.

https://github.com/syoyo/blender-usdz-test-scenes/tree/main/scenes/blendshapes

https://github.com/syoyo/tinyusdz/assets/18676/d552ce7f-0faa-4ba6-98a4-e8602aad6331

tydra_to_renderscene input.usd --dumpusd, then open USD file in Blender 4.1

[syoyo]

Extended Vulkan-glTF-PBR with TinyUSDZ Tydra to load USD asset.

Current status: Static mesh + diffuseColor texture works.

https://github.com/syoyo/Vulkan-glTF-USDZ-PBR

[syoyo]

Now rendermesh-refactor is merged into dev. https://github.com/syoyo/tinyusdz/commit/b41e1ddb6837e53827d9c234c0ded59190fa6a0b Further Tytra-related development will be done in dev branch.

[syoyo]

Now PBR shading starting to work!

https://github.com/syoyo/Vulkan-glTF-USDZ-PBR