twilio / rtc-diagnostics

Other
46 stars 14 forks source link

iPhone 12 and above not able to detect audioOutput #84

Open thlee1122 opened 1 year ago

thlee1122 commented 1 year ago

Describe the bug I utilized the "testAudioOutputDevice" function provided by the "@twilio/rtc-diagnostics" package to examine the audio output. This involved playing back a recorded audio input using the "testAudioInputDevice" function on Chrome and Safari browsers, specifically on iPhone 12 and newer models.

Just to give you some context, I developed a function called "playBack" that incorporates the "testAudioOutputDevice" function, as well as a "readAudioInput" function that uses the "testAudioInputDevice" function.

readAudioInput function:

const readAudioInput = useCallback(
  (options) => {
    if (audioInputTest) {
      audioInputTest.stop();
    }

    log.debug("AudioInputTest running");

    const duration = options.enableRecording
      ? RECORD_DURATION
      : INPUT_TEST_DURATION;
    options = { duration, ...options };
    audioInputTest = testAudioInputDevice(options);

    setIsAudioInputTestRunning(true);
    if (options.enableRecording) {
      log.debug("Recording audio");
      setTestEnded(false);
      setIsRecording(true);
      setWarning("");
      setError("");
    }

    audioInputTest.on(AudioInputTest.Events.Volume, (value) => {
      setInputLevel(getAudioLevelPercentage(value));
    });

    audioInputTest.on(AudioInputTest.Events.End, (report) => {
      if (playbackURI && report.recordingUrl) {
        URL.revokeObjectURL(playbackURI);
      }

      if (report.recordingUrl) {
        setPlaybackURI(report.recordingUrl);
      }

      setIsRecording(false);
      setIsAudioInputTestRunning(false);

      log.debug("AudioInputTest ended", report);
    });

    audioInputTest.on(AudioInputTest.Events.Error, (diagnosticError) => {
      log.debug("audioInputTest error", diagnosticError);
      setError(getErrorMessage(diagnosticError));
    });
    audioInputTest.on(AudioInputTest.Events.Warning, (name) => {
      log.debug("warning", name);
    });
    audioInputTest.on(AudioInputTest.Events.WarningCleared, (name) => {
      log.debug("warning-cleared", name);
    });
  },
  [playbackURI]
);

playBack function:

const playAudio = useCallback(
    (options) => {
      log.debug("AudioOutputTest running");

      options = { doLoop: false, duration: RECORD_DURATION, ...options }; // Set duration (which works as timeout) explicitly since testAudioOutputDevice is not ended in rare cases (i.e. iOS/Safari)
      const audioOutputTest = testAudioOutputDevice(options);

      setIAudioOutputTestRunning(true);
      setTestEnded(false);
      setWarning("");
      setError("");

      audioOutputTest.on(AudioOutputTest.Events.Volume, (value) => {
        setOutputLevel(getAudioLevelPercentage(value));
      });

      audioOutputTest.on(AudioOutputTest.Events.End, (report) => {
        setIAudioOutputTestRunning(false);
        setTestEnded(true);
        setOutputLevel(0);

        const stdDev = getStandardDeviation(report.values);

        if (stdDev === 0) {
          setError(translate("page.twilio.rtc.test.audio.error.title"));
        } else if (stdDev < AUDIO_LEVEL_STANDARD_DEVIATION_THRESHOLD) {
          setWarning(translate("page.twilio.rtc.test.audio.warning.title"));
        }

        log.debug("AudioOutputTest ended", report);
      });

      audioOutputTest.on(AudioOutputTest.Events.Error, (diagnosticError) => {
        log.debug("audioOutputTest error", diagnosticError);

        setError(getErrorMessage(diagnosticError));
      });
    },
    [translate]
  );

When I execute the "testAudioInputDevice" function, it successfully runs the audio input test and returns a report. Upon inspecting the report object, I observe the recordingUrl and an array of values containing numerical data. I store this recordingUrl as a React state and pass it to the "playBack" function when initiating the playback process.

However, when I examine the report generated by the "testAudioOutputDevice" function, it consistently returns a value array filled with zeros. As a result, this array of zeros signifies the absence of actual audio, leading to detection of "no audio".

Please refer to the report object screenshots below.

This works perfectly fine on iPhone 11 but it doesn't seem to work for iPhone 12 and above.

To Reproduce

  1. Initiate the readAudioInput function, which uses "testAudioInputDevice" function from "@twilio/rtc-diagnostics" to record the audio input. This will set recordingUrl as the React State.

  2. Initiate the playBack function, which uses "testAudioOutputDevice" function from "@twilio/rtc-diagnostics" by passing recordingUrl React State.

Expected behavior The report object that is returned by "testAudioOutputDevice" function should contain a value array with numbers. That way, we can successfully play back the recorded audio input to the user.

Screenshots Screenshot #1: audio input test report object image

Screenshot #2: audio output test report object image

Environment (please complete the following information): iPhone 12

iPhone 14

SDK Version: "@twilio/rtc-diagnostics": "^1.0.0"