USDZ export causes our models to be face-down and tiny

[sreyemnayr]

This relates to the AR Mode in Model Viewer on iOS.

We’re trying to sort out why our models are being flipped face-down and resized to be tiny only on iOS when the AR mode button is pressed, even though they appear correctly in the viewer’s normal mode and on Android devices.

An example: https://playcanvas.com/viewer?load=https://metadata.plaguepoppets.io/ipfs/bafybeichk75gele2kbns5ymcesq5ekjpiju2egdqf4pp6uyobfxcevn26m.glb

Video: https://github.com/playcanvas/model-viewer/assets/8558670/3b954a46-ee4c-441a-913d-24ba0f151c7f

Any ideas?

[slimbuck]

On iOS the model must be converted to usdz. My guess is our converter isn't taking some of the node scaling or skinning into account.

[sreyemnayr]

On iOS the model must be converted to usdz. My guess is our converter isn't taking some of the node scaling or skinning into account.

Is there a different repo I should raise the issue on?

[slimbuck]

The usdz code is in the engine, so actually yes please make an issue there. cc @mvaligursky

[mvaligursky]

Could you try this with a model that is not animated please? USDZ format support does not handle animations currently.

[sreyemnayr]

That's interesting... Apple's "Reality Converter" seems to support animations so I guess it's not an issue with the file format, just not implemented by playcanvas? I'll see if I can rip the animations out of one of our GLBs to see if it helps... not an ideal solution for us for sure, though.

Could you try this with a model that is not animated please? USDZ format support does not handle animations currently.

[mvaligursky]

There are many things missing from our USDZ implementation currently: https://github.com/playcanvas/engine/issues/4644

It's a complex format. They have native libraries to generate it, which are not available for javascript, so we generate a text based version of the format ourselves, but the documentation for this is limited.

[sreyemnayr]

Can confirm that removing animations does not solve the issue: https://playcanvas.com/viewer?load=https://metadata.plaguepoppets.io/ipfs/bafybeig25xsytquqq5fqkqccvsmyoamaudsct6dqop67xdk4r5ijo4oakm.glb

[sreyemnayr]

There are many things missing from our USDZ implementation currently: #4644

It's a complex format. They have native libraries to generate it, which are not available for javascript, so we generate a text based version of the format ourselves, but the documentation for this is limited.

Maybe I'll see if ChatGPT can help with porting the python libraries to javascript... There's also this C++ library from Google: https://github.com/google/usd_from_gltf

[mvaligursky]

The other option would be to convert your glTF models to USDZ off-line, and download and use those for QuickLook instead of using runtime conversion, which is more limited.