Skeletal animations in USDZs generated with usd_from_gltf no longer work in iOS 15 quicklook

[viperphase1]

We are using usd_from_gltf to generate usdz files. The usdz files with skeletal animations are working fine in quicklook on iOS 14 but not in iOS 15. You can test this with browser stack using this link: https://view.seekxr.com/nestbr/felix.html

Here are some related discussions:
https://developer.apple.com/forums/thread/691709
https://twitter.com/usdzshare/status/1463219078890283008

It seems Apple has once again changed the spec for USD files. Will this project be updated so that the source model glbs can be reprocessed to work on iOS 15 quicklook?

[julienbfabre]

Same problem here, i'd say this project is not directly affected though, @viperphase1 you problably need to update USD, that this project relies on, i tried that but of course the compilation on linux failed this time, but maybe you'll have more luck than i.

[julienbfabre]

actually i managed to install the latest USD version and the problem persists, objects converted with usd_from_gltf now show "Object requires a new version of iOS" on an up to date iOS 15.5 device, so i'm guessing usd_from_gltf also needs to be updated somehow. any luck @viperphase1 ?

[michaelgold]

Hey all. The only solution I found that supports skeletal animations is to use the usdzconvert scripts from Apple.

I built a docker container that uses the most recent version of usdzconvert (0.66) and the most recent version of USD (22.05b)

You can use it by running this command to convert your gltf/glb to a usdz file: docker run -v ${PWD}:/mnt/assets --rm michaelgold/usdzconvert /mnt/assets/your-gltf-file.glb

More info is here: https://github.com/michaelgold/usdzconvert-docker

I'll try to keep this repo updated as Apple and Pixar release new versions

[lenkawell]

Although Apple seems to have fixed the problem with RealityKit, the "invisible" model problem still exists with SceneKit apps and Xcode (which uses SceneKit). Does anyone know what is wrong with the USDZ model that is causing this and if so, how I might be able to detect it in our SceneKit app? This happens regularly because Sketchfab is still using this converter so any of our users who download USDZ models with skeleton animations and try to use them in our SceneKit app run into the issue. Thanks!

[lenkawell]

After debugging this problem further with SceneKit, I discovered that the all of the converted USDZ model skinners have their baseGeometryBindTransform matrix set to all zeros. A scale transform of zero results in an invisible model. So the solution was to detect this after loading a model and setting it to SCNMatrix4Identity. SceneKit then renders the model fine.

Another problem that I found is that all the skinners have their boneIndices SCNGeometrySource comprised of UInt16 values, which renders OK but SCNVIew.hitTest does not support. The solution for that is to convert the data for boneIndices to UInt8 which hitTest handles fine. Let me know if you're interested in the code to do the conversion.

[meshula]

Do you know where the zero matrices came from? ie. the original data, or one of the conversion steps?

[lenkawell]

I haven't analyzed the usd_from_gltf conversion code so I can't say for sure, but since the GLTFs from Sketchfab seem OK and the zero matrices seem to be in every USDZ file with a skeleton that I've looked at, I'm presuming the problem is with the conversion.