search

facebookarchive / react-360

Create amazing 360 and VR content using React
https://facebook.github.io/react-360
Other
8.73k stars 1.23k forks source link

AudioModule does not function on mobile (ios) even with user interaction html #652

Closed drubio1989 closed 5 years ago

drubio1989 commented 5 years ago

Description

I have a freshly created application in order to test audio on mobile (ios/ iphone SE). Based on WebAudio documentation and the react 360 issues regarding the issue, I've learned that I needed to create an html button entity to create 'user interaction' and then once that is achieved, the audio should work.

Expected behavior

Audio should play on mobile when a user clicks the html button from index.html

Actual behavior

Audio fails to play on mobile when a user clicks the html button from index.html

Reproduction

Here is the code:

index.html

<html>
  <head>
    <title>ExampleVR</title>
  <style>body { margin: 0; }</style>
<meta name="viewport" content="width=device-width, initial-scale=1, user-scalable=no">
</head>
  <body>
  <!-- Attachment point for your app -->

<div id="container">

  <button id="enter" onclick="enterApp();">
    Click to Enter VR
  </button>

</div>
<script src="./client.bundle?platform=vr"></script>
<script>
  // Initialize the React 360 application

  function enterApp() {
    React360.init(
      'index.bundle?platform=vr&dev=true',
      document.getElementById('container'),
      {
        assetRoot: 'static_assets/',
      }
    );  
  }

</script>

index.js

  export default class VRProject extends React.Component {
    playMusic() {
    AudioModule.playEnvironmental({
      source: asset('ambient.mp3'),
      volume: 1, // play at 3/10 original volume
    });
  }

render() {
  return(
    <View style={styles.panel}>
      <View style={styles.greetingBox}>
        <VrButton onClick={() => this.playMusic()}>
          <Text style={styles.greeting}>I am a button</Text>
        </VrButton>
      </View>
    </View>
  );
}

}

Solution

User interaction should solve the issue of playing audio, especially for ambient sound.

Additional Information

kevinsimper commented 5 years ago

VrButton is not a html button, but a button rendered to a WebGL canvas :)

Try to make a demo of a audio file on an iPhone without WebVR and see how it works, because then you will see it doesn't work and you can eliminate that it doesn't have anything to do with ReactVR and you can take it from there 😄

drubio1989 commented 5 years ago

Hey @kevinsimper thanks for putting your input too on this issue, I appreciate it. I'll try what you recommend but take a look at my index.html file. From the other issue you were commenting on, I went ahead and actually created the html button as seen here like you said:

        <button id="enter" onclick="enterApp();">
            Click to Enter VR
        </button>

When the button is clicked, the enterApp(); loads my my react 360 code as below:

function enterApp() {
    React360.init(
      'index.bundle?platform=vr&dev=true',
      document.getElementById('container'),
      {
        assetRoot: 'static_assets/',
      }
    );  
  }

Are you telling me that this is still not the way to go because that's what I thought what was needed to happen? I'll go ahead and try the other recommendation too.

kevinsimper commented 5 years ago

The click on html button and the audio Play event has to happen in the same action on Smartphones, but as said, make a demo without react360.

drubio1989 commented 5 years ago

@kevinsimper I've tried my best to understand the problem and create a barebones html file like you've recommended but I'm still at a lost as to why the audio will not work at all or what to take away from this. Here is the basic deployed html file link I've constructed audio.

The html makes use of the audio html element and works on both desktop and mobile. I'm afraid I'm just not getting it though. I still don't get the implication of this. When you said,

"the click on html button and the audio Play event has to happen in the same action on Smartphones, "

Again, this is what I have interpreted from what you've said:

      `index.html

       <div id="container">
          <button id="enter" onclick="enterApp();">
               Click to Enter VR
          </button>

          <audio style="display:none" id="ambient" controls src="./static_assets/ambient.wav"></audio>
       </div>

           <script>

             function enterApp(){
               React360.init(
                 'index.bundle?platform=vr&dev=true',
                  document.getElementById('container'),
               {
               assetRoot: 'static_assets/',
               }
             );

             let media = document.getElementById('ambient');
             media.play();
           }
           </script>

The button that is presented creates the React 360 application and triggers the audio. This works on both desktop and mobile.

But now when the audio clip is finished, I have no way of playing it again unless I make another html button. So what does this mean? I can't play audio without a user clicking a real html button? The html buttons clobber all the react 360 rendered code and looks really bad. I've tried through a NativeModule to get the document information and to programmatically play and stop the audio but that doesn't work. I thought it would have worked because I read somewhere on another thread #573 with @andrewimm commenting that once the user has had some sort of interaction with the window, the audio should work on the react side. It doesn't.

I've tried to use the docs to create an audio, modify it, stop it, etc. and the AudioModule just doesn't work on mobile. Neither can I programatically attach methods onto onClick handlers from a nativeModule similar to the way that the BrowserinfoModule works with the document.

@kevinsimper do you have a working sample among your repos? I thought I would try to find some in your profile but you got a bunch :/. If you've got anymore insight to share, I'd appreciate it. I haven't found a single place in react 360 github forums or on stackoverflow where audio was played on mobile. It seems to be harder than I thought.

kevinsimper commented 5 years ago

@drubio1989 You can read this article 👍 https://bitmovin.com/play-not-play-new-autoplay-policies-safari-11-chrome-64/

drubio1989 commented 5 years ago

Thanks for your help in this, I've read it but even the AudioModule.playOneShot doesn't work on mobile and that has nothing to do with autoplay. I'm not sure there is a way to have react 360 audio work on mobile as of right now :/ I'll just see what I can do at this point.

ansible-comme commented 5 years ago

Hi,

@kevinsimper thanks for the article share 👍

@drubio1989 I've been trying to do this for a while and following this and other threads. Now... this code is working for video & sound on mobile - tested on android & ios.

index.html

    <div id="container">
      <div id="buttonHold">
      <button id="playbutton">
        Enable Sound for Mobile & Begin VR
      </button>
      <video id="video" width="0" height="0">
        <source src="video.mp4" />
      </video>
      </div>
    </div>

index.html Githubissues.

  • Githubissues is a development platform for aggregating issues.