Hear yourself when speaking (with headphones)

[toomim]

Old wired telephones will loop the speaker's mic signal back through the earpiece, so the speaker can hear themselves speak. This gives the speaker a sense of how their voice is being picked up by the microphone, and sent to the other side. Humans naturally use their own ears as feedback for how their voice is being received. (Consider that someone wearing earplugs is likely to speak too loud, because they are listening to their own voice, and can't tell how loud it is.) Musicians also set up a special monitor speaker or headphones to hear themselves, when live on stage. This feedback is critical for knowing how you're received.

Many of the bugs in video chat are due to not knowing how you are being received. We sometimes have our microphones off, and try to talk, without knowing that the other party can't hear us. We often have our mics ON while there's loud background noises, that distract everyone else on the chat without us realizing. We might breathe into the mic, or rub it with our fingers, and creating gross scratching noises that everyone else hears blare and we ourselves are unaware of, because there's no monitor for our audio.

Internet/cell phone audio chat and cell phones have stopped looping the mic audio back through the earpiece to prevent feedback. It's harder to cancel out feedback when you aren't sure how much latency there is on the signal (vs. analog wires), and when you're sending audio over the internet, you don't know how much latency there will be. So they just turn it off.

However, if you're using headphones—which any serious video chatter should—then there's no feedback possible, and it's very useful to have a monitor again. Serious video chatters would benefit from a mic monitor along with nice headphones.

So we should implement a mic monitor loop that's available to people wearing headphones. I'm not sure what the UI design should be, yet. Perhaps we can start with just an advanced feature (accessed via the console or a tiny menu or keybinding) and then figure out the UI after we experiment with it.

[toomim]

I think I know how to overcome the #1 obstacle that prevents other videochat solutions from implementing a monitor.

The main obstacle to a monitor is preventing feedback. If we assume our users are experts, we can let them control when the monitor is off and on, and they can ensure they only enable it when using headphones. However, if they accidentally unplug their headphones without disabling the monitor, or turn it on without plugging in headphones, they'll get super annoying loud feedback that disrupts everyone around them. Even if such occurrences are rare, they are so painful that videochat designs try to avoid them at all costs — disabling monitoring completely.

The solution employed in music software (like Garageband) is to detect feedback automatically, and disable the monitor whenever feedback is detected. However, this has two drawbacks:

1. It must waits for feedback to happen for a ~second before noticing it and turning it off, so the user still perceives it
2. The user has to then manually turn the monitor back on, once they solve the feedback problem (e.g. by plugging in headphones)

We can solve these problems by actively testing for feedback before the user can hear it. Speakers can transmit tones at frequencies that the human ear cannot detect—22khz—and microphones can pick these up. So we can make the laptop transmit a 22khz signature periodically during a videochat, and if it ever detects the sound coming back through the microphone, it will know that there is feedback possible, and it can disable the monitor. Otherwise it can show a little headphones icon to show that headphones are detected and the monitor is enabled.

This will enable tawk.space to be the first video chat system with a great automatic monitor, that lets people hear what other people can hear, and self-regulate their self-presentation.

[karth295]

This's pretty cool!

You might be able to use the label field of this thing: https://developer.mozilla.org/en-US/docs/Web/API/MediaDeviceInfo

navigator.mediaDevices.enumerateDevices()
.then((arr) => console.log(JSON.stringify(arr)));

Here's what I get without headphones:

[
  {
    "deviceId": "default",
    "kind": "audioinput",
    "label": "Default",
    "groupId": "6e24e7024206ad48f573f0766abb2bb7d35c542b450ee25e495aa03dad32cb8c"
  },
  {
    "deviceId": "0bccd9f5684f5091606134506eca908e8bbc423de27825f92a67963a17dfbae9",
    "kind": "audioinput",
    "label": "Built-in Audio Analog Stereo",
    "groupId": "14598e9c298716dd7f4e67523537e35d06fc9c9958b11cb32a13895cf6ab1507"
  },
  {
    "deviceId": "4f92d35334e6b0b786c528577a3523c3eaa25f3715226cd226405a10deeca276",
    "kind": "videoinput",
    "label": "Lenovo EasyCamera (5986:0295)",
    "groupId": ""
  },
  {
    "deviceId": "default",
    "kind": "audiooutput",
    "label": "Default",
    "groupId": "default"
  },
  {
    "deviceId": "fb127b9a4908502a79db1ee864b3a1d0f163db11db4de5811ad233d84c98cbc8",
    "kind": "audiooutput",
    "label": "Built-in Audio Analog Stereo",
    "groupId": "fb127b9a4908502a79db1ee864b3a1d0f163db11db4de5811ad233d84c98cbc8"
  }
]

I don't have headphones here to test, but I bet the label will indicate that the device is headphones.

If not, we can still use the deviceId to remember whether a particular device the user is using is headphones or not.

[toomim]

Oh wow, very cool.

I doubt this can distinguish, though, between headphones and speakers plugged into the headphones port.

[morgandixon]

You could figure it out by emitting a brief tone from the speakers and testing if it's audible from the microphone.

Sent from my iPhone

On Feb 8, 2017, at 9:30 AM, Michael Toomim notifications@github.com wrote:

Oh wow, very cool.

I doubt this can distinguish, though, between headphones and speakers plugged into the headphones port.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[toomim]

Yep, that's what I posted two comments ago.