search

googlevr / gvr-android-sdk

Google VR SDK for Android
http://developers.google.com/vr/android/
Other
3.28k stars 1.28k forks source link

Does VrVideoView support spatial audio? If not, how can I make it enabled when playing 360 video with cardboard sdk? #103

Closed lingkang1988 closed 8 years ago

Ood-Tsen commented 8 years ago

VrVideo using ExoPlayer library. So it depends on the codec capability and extractor in ExoPlayer.

FirmianaRain commented 8 years ago

Hi, can we replace ExoPlayer with other player, such as ijkplayer?

Ood-Tsen commented 8 years ago

I didn't dig it in. At least you got to rewrite a VrVideoPlayerInternal by yourself.

nathanmartz commented 8 years ago

VR Video View does not currently support spatial audio as part of the movie payload. However, the SDK includes APIs for spatial audio as well as examples for how to use them. You can combine these with a Video View.

lingkang1988 commented 8 years ago

Thanks, nathanmartz. I know the SDK includes APIs for spatial audio, however the APIs only accept audio files instead of video files. So if I use google sdk to enable spatial audio when playing video, I need to prepare a separate audio file which is not that accepted.

So I have the following two questions:

  1. How does the SDK implement spatial audio? With libraries such as OpenSL ES and Open AL or audio library written by Google? I tried Android OpenSL ES, it seems has no supports for 3D audio(game profile).
  2. If I want to use spatial audio in video playing with Google VR SDK, what is the best practice(as detailed as possible)?
jkammerl commented 8 years ago

The Google VR SDK on Android supports the decoding and binaural rendering of an ambisonic soundfile. You could separately trigger spatial audio playback together with the video playback, but then the synchronization of both media streams will be quite challenging. Android's Exoplayer and the spatial audio engine have very different buffering schemes and signal paths, so unfortunately I don't think this is easily possible. However, I agree that this is an interesting feature which is worth considering in a future release.

niusounds commented 8 years ago

👍

markya0616 commented 8 years ago

Yes, we also need this feature!!!! Can we use the left and right audio channel to create the spatial audio effects?