hackingbeauty / react-mic

Record audio from a user's microphone and display a cool visualization.
https://hackingbeauty.github.io/react-mic/
450 stars 156 forks source link

Browser audio recording is not supported #147

Open manosec opened 9 months ago

manosec commented 9 months ago

Hi there,

I was working with react-mic to stream the audio buffers to backend socket event handlers in NodeJS, my goal is to transcribe the audio to text.

I'm able to record the audio and stream it to the socket events but when I deploy the app, I'm unable to record the audio. A pop-up says browser does not support audio recording. I have attached the reference screenshots below.

  1. When deployed in the server audio_notsupport

  2. In my local machine audio_succ

manosec commented 9 months ago

I have attached my react component code here,

import { ReactMic } from 'react-mic';
import React, { useState, useEffect, useRef } from 'react';
import io from 'socket.io-client';

const Example = () => {
  const [record, setRecord] = useState(false);
  const socketRef = useRef(null);
  const [result, setResult] = useState("")

  useEffect(() => {

    return () => {
      if (socketRef.current) {
        socketRef.current.disconnect();
      }
    };
  }, []);

  const downsampleBuffer = function (buffer, sampleRate, outSampleRate) {
    if (outSampleRate == sampleRate) {
      return buffer;
    }
    if (outSampleRate > sampleRate) {
      throw 'downsampling rate show be smaller than original sample rate';
    }
    var sampleRateRatio = sampleRate / outSampleRate;
    var newLength = Math.round(buffer.length / sampleRateRatio);
    var result = new Int16Array(newLength);
    var offsetResult = 0;
    var offsetBuffer = 0;
    while (offsetResult < result.length) {
      var nextOffsetBuffer = Math.round((offsetResult + 1) * sampleRateRatio);
      var accum = 0,
        count = 0;
      for (var i = offsetBuffer; i < nextOffsetBuffer && i < buffer.length; i++) {
        accum += buffer[i];
        count++;
      }

      result[offsetResult] = Math.min(1, accum / count) * 0x7fff;
      offsetResult++;
      offsetBuffer = nextOffsetBuffer;
    }
    return result.buffer;
  };

  const startRecording = () => {
    setRecord(true);
    const socket = io('http://localhost:3000');
    socket.on('connect', () => {
      console.log('Socket connected:', socket.connected);
      socketRef.current = socket;
    });

    socket.on("result", data=>{
      setResult(data)
    })
  };

  const stopRecording = () => {
    setRecord(false);
    if (socketRef.current) {
      socketRef.current.disconnect();
    }
  };

  const onData = async (recordedBlob) => {
    if (socketRef.current && socketRef.current.connected) {
      // const data = await downsampleBuffer(recordedBlob, 44100, 16000)
      // console.log(data)
      // console.log(recordedBlob)
      socketRef.current.emit('binaryData', recordedBlob);
    } else {
      console.log('Socket not connected');
    }
  };

  const onStop = (recordedBlob) => {
    console.log('recordedBlob is: ', recordedBlob);
  };

  return (
    <div>
      <ReactMic
        record={record}
        className="sound-wave"
        onStop={onStop}
        onData={onData}
        strokeColor="#000000"
        backgroundColor="#FF4081"
      />
      <button onClick={startRecording} type="button">
        Start
      </button>
      <button onClick={stopRecording} type="button">
        Stop
      </button>
    </div>
  );
};

export default Example;
manosec commented 9 months ago

Additionally, how can I modify the audio buffer here, I tried to downsample the buffer to make it compatible with Speech to Text service I'm using. I have attached the format below that I wanted the audio buffer convert to. Please share the relevant resources on this.

const {  samplesPerSecond, bitsPerSample, channels } = {
    "samplesPerSecond": 16000,
    "bitsPerSample": 16,
    "channels": 1
    };