Play spatial audio from ExoPlayer

[xdf103]

Can GVR play spatial audio (FOA) from ExoPlayer?

[jkammerl]

Currently, we don't support FOA playback from ExoPlayer.

[xdf103]

We can extract the 4 channel FOA audio data from ExoPlayer. Is there any way to pass this audio data stream to Google VR SDK to play out spatial audio?

[jkammerl]

The Google VR Android API doesn't support the processing of raw PCM audio buffers at the moment. Note that the binaural stereo output also would need to be played back via the Exoplayer buffer queues to ensure synchronization with the video stream.

[xdf103]

Is Google going to open source the native methods, e.g. nativeCreateSoundfield? If yes, where can I find the source code? I would like to know how the soundfield function is implemented.

[xdf103]

I noticed that VrVideoView uses ExoPlayer to read from video. Why not add a TrackRenderer to render spatial audio if the video contains ambisonic audio? Is it very difficult to add this feature or is there any limitation?

[jkammerl]

Using the TrackRenderer interface to render spatial audio in the Exoplayer should work. It good to know that there is demand for such feature and we might consider it in future SDK updates. The Google VR SDK is not open-sourced.

[dcower]

@xdf103 We use that approach to enable spatial audio rendering in YouTube Android and similar apps. Could you expand a bit on what you're looking for here? E.g., are you looking for spatial-audio-only playback or 360-video-with-spatial-audio playback? Do you need streaming video/audio? Feel free to reach out privately ([my username]@google.com) if you prefer.

[xdf103]

We are playing a 360 video with spatial audio (first-order ambisonic ACN SN3D Ambix format) using exoplayer. The video part is working fine and I can also hear the audio sound, but the sound is not spatial. I am thinking of implementing a custom TrackRenderer to render spatial audio. Are there any reference or documents for rendering spatial audio?

[gtStation]

@xdf103 have you happened to have found the resources to start your Spatial audio rendering? We are in need of a similar solution.

[jkammerl]

The upcoming GVR release will contain an API that enables the rendering PCM buffers of ambisonics and standard 5.1 surround content into real-time head-tracked binaural stereo. This API can for instance be wrapped into an AudioTrackRenderer for the Exoplayer to binaurally decode 360 video with ambisonics. Stay tuned.. it will become available soon.

[gtStation]

@jkammerl - that is great to hear.

Will this be included in the Unity version of the plugin? What about ambisonics for iOS? Which platforms can I use GVR SDK in terms of audio? Will the spatializer work on Oculus Rift/HTC, (PlayStation??)

[andrewlewis]

Regarding support for spatial audio rendering in ExoPlayer: we recently pushed an ExoPlayer GVR extension to our development branch. It provides GvrAudioProcessor which wraps GvrAudioSurround from the GVR SDK. There's some information on how to use this on our blog.

[markusekblad]

@andrewlewis @dcower Does this apply to the VrVideoView or is it only for building your own video player? I'm exploring how to add spatial audio to our VR video app which uses the VrVideoView. One solution is to go with the Facebook/TwoBigEars engine for spatial audio but then you have to have a separate sound file and sync the players which is not ideal.

[fashioncj]

@andrewlewis Can you provide a sample demo for GvrAudioProcessor. I can't find a override function buildAudioProcessors on new SimpleExoPlayer(){}. My extentions version is compile 'com.google.android.exoplayer:exoplayer:r2.4.3' and compile 'com.google.android.exoplayer:extension-gvr:r2.4.3'

[andrewlewis]

@fashioncj We moved the method to DefaultRenderersFactory, so the new pattern is to subclass DefaultRenderersFactory (or implement RenderersFactory) and pass it to ExoPlayerFactory.newSimpleInstance.

[sani3854]

How to youtube video play in VrVideoView (google)?

my code is :+1:

VrVideoView.Options options = new VrVideoView.Options(); //options.inputType = VrVideoView.Options.TYPE_STEREO_OVER_UNDER; // videoWidgetView.loadVideoFromAsset("congo.mp4", options);

                options.inputFormat = VrVideoView.Options.FORMAT_HLS;
                options.inputType = VrVideoView.Options.TYPE_MONO; // for 360 video

                videoWidgetView.loadVideo(Uri.parse("https://www.youtube.com/watch?v=H6SsB3JYqQg"), options);
                videoWidgetView.playVideo();

Error is Error fetching video manifest com.google.android.exoplayer.ParserException: Failed to parse the playlist, could not identify any tags.

Any solutions???