search

RenderHeads / UnityPlugin-AVProVideo

AVPro Video is a multi-platform Unity plugin for advanced video playback
https://www.renderheads.com/products/avpro-video/
232 stars 28 forks source link

AudioOutput+Facebook360 Audio #773

Open ColinNorthwayMOR opened 3 years ago

ColinNorthwayMOR commented 3 years ago

It would be great if AudioOutput was compatible with Facebook360 audio so I could spacialize the audio in the scene in some cases (my case: as you walk closer to the video it becomes less spacialized in the scene and more ambisoic and as you walk away the reverse is true)

AndrewRH commented 3 years ago

Hi,

What do you mean that the video becomes less spatialised / more ambisonic?

Thanks,

ColinNorthwayMOR commented 3 years ago

Yeah this terminology is confusing. I'll define the terms I'm using here with our concrete use-case.

You are facing north, there is a 360 video of a phone ringing. In the video, the phone is to the east.

We have two behaviours depending on how far away you are from the video, which is projected on a sphere.

If you are standing inside the sphere then the audio is Ambisonic - The phone audio sounds like it's coming from the East.

If you are standing 10 meters south of the sphere the audio is Spacialized - The phone sounds like it is coming from the North.

Ambisonic audio is baked into the video and is played back via the Facebook360 API (this is working great for us btw!)

Spacialized audio is played back through the Unity AudioSource.

Previous to using the Facebook360 API we separated the audio track from the video and played it back through an AudioSource using the Steam Audio API to "Abisonicify" the audio. We could then use the AudioSource "Spacial Blend" variable to slide between 3d (when you are far away from the video) to 2d (when you are inside the video).

We prefer the Facebook360 API because we don't have to separate the audio from the video for ambisonic playback. But we have lost the ability to slide between Spacialized (3d) and Ambisonic (2d) audio.

Playing a 360 video on a sphere 10 meters away might sound like a weird thing to want to do but we are a multiplayer space so your friend might start up a video and we want you to be able to see what they are doing. We actually change the shape of the sphere so it "envelops you" as you get closer, it's all very fancy. But now if you are visiting the museum with your friend and they start a video across the room you hear the phone ring in the wrong place.

Playing back through an AudioSource would also let us use the AudioSource volume distance-fall-off graph. Currently we just manually change the volume depending on your distance, which is fine, this would just be a nice-to-have.

Colin

On Tue, 18 May 2021 at 06:22, Andrew @.***> wrote:

Hi,

What do you mean that the video becomes less spatialised / more ambisonic?

Thanks,

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/RenderHeads/UnityPlugin-AVProVideo/issues/773#issuecomment-843167357, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALZT3MC7QMU2FFNSADYZA2DTOJSZ3ANCNFSM44XCUYPA .