PixarAnimationStudios / OpenUSD

Universal Scene Description
http://www.openusd.org
Other
6.12k stars 1.22k forks source link

Cannot Play Embeded Audio when Open USDZ #2759

Closed michsiu closed 1 year ago

michsiu commented 1 year ago

Description of Issue

I have some USDZ files and want to add sound to them. After following the Open USD api docs and using usdzip, the sound was added successfully but there is still no sound when I open the USDZ file

Steps to Reproduce

  1. uncompress USDZ file by changing ext name to zip and get usdc file
  2. add audio prim on usdcfile through python, the code is below
    
    from pxr import Usd, UsdGeom

stage_ref = Usd.Stage.Open('1.usdc')

prim = stage_ref.DefinePrim('/audio','Xform') print(prim)

the doc I followed is `https://developer.nvidia.com/usd/tutorials#:~:text=To%20define%20a%20new%20prim%20with%20a%20type%20use%20stage_ref.DefinePrim(path%2C%20type_name)`

3. add audio information to `usdc file` through `python`
the code is below

from pxr import Usd, UsdGeom, UsdMedia, Sdf, Vt

stage_ref = Usd.Stage.Open('1.usdc')

prim = stage_ref.GetPrimAtPath('/audio')

audio_prim = UsdMedia.SpatialAudio(prim)

audio_filepath_attr = audio_prim.CreateFilePathAttr("Tyrannosaur.mp3") aural_mode_attr = audio_prim.CreateAuralModeAttr("nonSpatial")
playback_mode_attr = audio_prim.CreatePlaybackModeAttr("loopFromStartToEnd")

stage_ref.Export('4_long.usdc')


the doc I follow is `https://openusd.org/release/api/class_usd_media_spatial_audio.html`
4. using OpenUSD `usdcat` tool to check if the audio information was added successfully
` usdcat C:\Users\download\3dtest\0-Tyrannosaur\1.usdc`
the result is below
![image](https://github.com/PixarAnimationStudios/OpenUSD/assets/18113569/70708d95-8e1a-42c2-8d8f-9fb2bc618286)

5. using OpenUSD `usdzip` tool to zip all files including mp3 file, and the result shows that the sound was add successfully
![image](https://github.com/PixarAnimationStudios/OpenUSD/assets/18113569/9d4d527f-e6e3-41d7-961c-f70ad27da764)

This is the `usdzip` command line on x64 dev command tool
`C:\Users\download\3dtest\0-Tyrannosaur>usdzip -a C:\Users\download\3dtest\0-Tyrannosaur\image2.jpg  -a 4_long.usdc -l -v image5.usdz`

This is the command line result
![image](https://github.com/PixarAnimationStudios/OpenUSD/assets/18113569/9fb8854b-7272-4c6b-9028-179b4561e2c3)

the result usdzfile is below ( you should change file name from`.zip` to `.usdz`)
[image5.zip](https://github.com/PixarAnimationStudios/OpenUSD/files/13171415/image5.zip)

the original usdzfile is below ( you should change file name from`.zip` to `.usdz`)
[Tyrannosaur.zip](https://github.com/PixarAnimationStudios/OpenUSD/files/13171418/Tyrannosaur.zip)

The Result usdc file is below ( you should change file name from`.zip` to `.usdc`)
[4_long.zip](https://github.com/PixarAnimationStudios/OpenUSD/files/13171431/4_long.zip)

### System Information (OS, Hardware)
win10
### Package Versions
python 38
latest Open USD
### Build Flags
N/A
asluk commented 1 year ago

While OpenUSD has a UsdMediaSpatialAudio schema, it does not currently ship with a runtime that consumes that data to play back audio. Similarly OpenUSD has UsdPhysics schemas for rigid body dynamics, but does not ship with a physics engine that consumes that data. I am not sure which products based off of USD support spatial audio right now-- NVIDIA Omniverse does, but needs to reconcile its audio schema with UsdMedia. Apple ARKit-based runtimes might work.

michsiu commented 1 year ago

Hello @asluk , I checked the doc again and resolved the issue, the audio works now!

According to the doc, the SpatialAudio prim should be placed under Xform prim and set the playback mode as "loopFromStage", it works now