Sometimes audio noise can happen at the moment when zooming in an object

takahirox commented 4 years ago

Description

Sometimes, not always, audio noise can happen at the moment when zooming in an object if someone in the room is speaking.

To Reproduce

Enter a room
Run a bot with audio node run-bot.js -a bot-recording.mp3 -r roomId
Place an asset like a picture anywhere in the room
Press f on the asset and zoom in
Confirm that an audio noise can happen. If it doesn't, repeatedly press until it happens

If the audio noise doesn't happen, move close to or far from the bot and try again. It seems this problem very depends on the distance from the speaker or audio volume.

Expected behavior

Audio noise doesn't happen.

Hardware

Device: [e.g. Desktop, phone, VR headset] Desktop
OS: [e.g. Windows, iOS, Linux] Windows
Browser [e.g. Firefox] Firefox and Chrome

Additional context

In my environment, it happens both on Firefox and Chrome but I feel it happens more easily on Chrome? No confidence tho.
As I wrote in "To reproduce", this problem seems very depending on the distance from the speaker or audio volume. In my test, it may happen more easily if I am closer to the bot?
It doesn't seem to happen at the moment when you are back to normal from zoom in mode.

johnshaughnessy commented 4 years ago

@takahirox I have only been able to reproduce when after using /audiomode to switch the default preference. I could reproduce this a few days ago, but now can only reproduce it for media (not for avatar voices). My guess is that https://github.com/mozilla/hubs/pull/3090 fixed it for avatars, and now we need to merge https://github.com/mozilla/hubs/pull/3118 to fix it for other media.

johnshaughnessy commented 4 years ago

Unrelated, I was unable to use the bot script with node run-bot.js -a bot-recording.mp3 -r roomId. I used a roomId from the dev server (what we connect to for local development). Am I doing this wrong?

[jomb@jlomb2 bot]$ node --trace-warnings run-bot.js -a bot-recording.mp3 -r yLSot4W
(node:1498052) UnhandledPromiseRejectionWarning: Error: socket hang up
    at connResetException (internal/errors.js:612:14)
    at Socket.socketOnEnd (_http_client.js:493:23)
    at Socket.emit (events.js:326:22)
    at endReadableNT (_stream_readable.js:1244:12)
    at processTicksAndRejections (internal/process/task_queues.js:80:21)
    at emitUnhandledRejectionWarning (internal/process/promises.js:168:15)
    at processPromiseRejections (internal/process/promises.js:247:11)
    at processTicksAndRejections (internal/process/task_queues.js:94:32)
(node:1498052) Error: socket hang up
    at connResetException (internal/errors.js:612:14)
    at Socket.socketOnEnd (_http_client.js:493:23)
    at Socket.emit (events.js:326:22)
    at endReadableNT (_stream_readable.js:1244:12)
    at processTicksAndRejections (internal/process/task_queues.js:80:21)
(node:1498052) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
    at emitDeprecationWarning (internal/process/promises.js:180:11)
    at processPromiseRejections (internal/process/promises.js:249:13)
    at processTicksAndRejections (internal/process/task_queues.js:94:32)
[jomb@jlomb2 bot]$ node --version
v14.8.0

takahirox commented 4 years ago

@johnshaughnessy Thanks for the comment. I still can reproduce the noise problem even with #3090. So First I'd like to confirm if we see/saw the same noise problem. I recorded the problem and put the video in Discord (and zip archived one here noise.zip). Can you check it?

Regarding run-bot.js, I think the command looks fine. Does it not work even without -a option? I think it may be better to open an issue for that.

johnshaughnessy commented 4 years ago

I see now -- this is a different problem. I was able to reproduce it and saw the same thing you did.

Maybe the problem has to do with how the audioListener is re-parented in the camera-system after the mode changes (e.g. from CAMERA_MODE_FIRST_PERSON to CAMERA_MODE_INSPECT).

In the lobby, in third person mode, or in first person mode, you should hear sounds from wherever the viewing-camera is, so the camera-system ensures that the audioListener is normally under the viewing-camera node. However, when you are in CAMERA_MODE_INSPECT, your viewing-camera will be up close to the object, but you should be hearing sounds where your avatar-pov-node (your avatar's head) is instead.

At the end of the camera-system's tick, we ensure that the audioListener is parented under the correct node (either viewing-camera or avatar-pov-node): https://github.com/mozilla/hubs/blob/master/src/systems/camera-system.js#L502

However, this reparenting should probably happen before the viewing-camera ever moves (before orbit or moveRigSoCameraLooksAtPivot or uninspect or inspect), so that the audioListener's updateMatrixWorld does not set panner node properties: https://github.com/MozillaReality/three.js/blob/hubs/master/src/audio/PositionalAudio.js#L119

Hubs-Foundation / hubs

Sometimes audio noise can happen at the moment when zooming in an object #3098