search

webaverse / app

Web metaverse client
https://app.webaverse.com
MIT License
338 stars 212 forks source link

Avatar grunts breaks speech phoneme recognition #3651

Closed tcm390 closed 1 year ago

tcm390 commented 1 year ago

The issue is caused by that both audioWorker and microphoneWorker are setting the value of this.volume

related: https://github.com/webaverse/app/issues/3645

Result:

https://user-images.githubusercontent.com/60634884/185251426-803f8f20-16d5-4c41-9b26-8f746012d6e0.mp4

tcm390 commented 1 year ago

disable audio worker after grunt end

Can we disable audio worker after the grunt effect is completed?

If volume goes from 1 -> 0 in one frame won't this cause it to get stuck? That's probably an even worse effect than the bug it is fixing.

How about choose which one to emit when it's enabled, not just by chance when it's zero?

avaer commented 1 year ago

Can we disable audio worker after the grunt effect is completed? No I don't think that's right, creating/destroying workers is a pretty heavy operation to run on every sound effect...

tcm390 commented 1 year ago

Can we disable audio worker after the grunt effect is completed? No I don't think that's right, creating/destroying workers is a pretty heavy operation to run on every sound effect...

disable audioWorker to set the volume when isGrunting is false

Got it. Is it okay to use isGrunting in avatar.js to decide whether audioWorker is allowed to set volume or not?

avaer commented 1 year ago

Got it. Is it okay to use isGrunting in avatar.js to decide whether audioWorker is allowed to set volume or not?

Probably not, because I think multiple grunts can happen at a time. At the very least I think this would need to be a stack number, not a boolean.

And if this is related to manuallySetMouth, shouldn't both be in the same class? Currently it's spread around so the code is confusing to read.

tcm390 commented 1 year ago

And if this is related to manuallySetMouth, shouldn't both be in the same class? Currently it's spread around so the code is confusing to read.

disable audioworker set volume after grunting

Got it. Renamed characterBehavior.manuallySetMouth to characterBehavior.disableAudioWorkerSetVolume. And used characterBehavior.disableAudioWorkerSetVolume to disable audioWorker after emitting grunt.

avaer commented 1 year ago

It looks like this is still using the disable name: https://github.com/webaverse/app/pull/3651/files#diff-fcb2093ac11d9b04a179d3dedbef5c0aae4d14b7f71db90e483becca4170b7ccR2041

tcm390 commented 1 year ago

It looks like this is still using the disable name: https://github.com/webaverse/app/pull/3651/files#diff-fcb2093ac11d9b04a179d3dedbef5c0aae4d14b7f71db90e483becca4170b7ccR2041

Sorry Changed it. change disableAudioWorkerSetVolume to enableAudioWorkerSetVolume