search

facebookarchive / react-360

Create amazing 360 and VR content using React
https://facebook.github.io/react-360
Other
8.73k stars 1.23k forks source link

Audio format documentation #351

Closed jgwinner closed 6 years ago

jgwinner commented 7 years ago

Description

In the documentation https://facebook.github.io/react-vr/docs/sound.html mentions: You can read more about supported audio formats in the docs for the Sound Effects API.

That page, however, just identifies API's; getSupportedResource seems to be the only one and that doesn't seem to discuss what works. I would at least change this in the docs; it would be better to say something like "Mono files are best, to allow 3D positioning" or the like.

Expected behavior

It looks like only Mono files are supported - if this is generally true (to permit 3D localization), it would be great if that could be documented. This may be a browser specific thing but my guess is that it's pretty common.

It looks like this may also be under-documented in three.js, but let's make it easier to adopt ReactVR.

Actual behavior

You get an error if you load a stereo .wav file, at least on Windows.

Reproduction

Load a stereo wave file via VrSound, you get an error:

Failed to fetch audio: ../static_assets/freesound278205ianstargem__switch-flip-1.wav VRAudioBufferSource.js:46:8 The buffer passed to decodeAudioData contains invalid content which cannot be decoded successfully.

The file is at: https://freesound.org/people/IanStarGem/sounds/278205/

Solution

If only mono is loaded, just mention it.

Additional Information

andrewimm commented 7 years ago

Hmm, I could swear there used to be explicit documentation on version formats. I think it got changed out when the audio APIs got more modularized: https://github.com/facebook/react-vr/blob/master/ReactVR/js/Audio/getSupportedFormats.js

Our audio support isn't tied to Three in any way, it's all implemented with standard browser Web Audio APIs: https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API WAV is supported. It just sounds like that particular WAV file is corrupt in some way that browsers can't load it.

Stereo channels should still be supported for certain cases, but I do agree that anything being used for positional audio (<Sound>) should be mono.

jgwinner commented 7 years ago

Thanks Andrew!

It might be browser specific as I found it on a ReactVR demo site! So it's really odd it didn't work in FireFox nightly (generated the above error).

I'll put back the stereo version and try it in GearVR.

I was using it in VrSound (love the lower case "R" ... makes sense from a naming convention but it's hard for me to type Vr, lol), so for that mono may be a requirement.

I think it would be nice if the loader just took one channel instead of giving an error? Is that an option?

For now, I think best practice would be just to use Mono and that's what I'll recommend in the book, but I'll leave it open.

andrewimm commented 7 years ago

I do not think that the problem is that the audio is stereo. I believe it's because the audio fails to load in the browser before it gets to the processing phase.

andrewimm commented 7 years ago

A quick google search finds a number of bugs reporting audio that works on IE and Chrome, but fails to load in FF. Just seems like a browser issue.

jgwinner commented 7 years ago

It loaded fine in Audacity with no errors. When I converted it to Mono, it played fine.

I can check some other way, any ideas? Maybe I could just make an HTML page with a sound on it. I'll do that and check.

jgwinner commented 7 years ago

Yup - the Stereo version worked OK in Chrome (I didn't realize Chrome did WebVR yet, thought I had to go to Chromium. This stuff moves fast).

So, this is a FF bug. You can close it, although an enhancement to convert to right channel on failure to load would be cool.

I'll test more browsers next time.