Voice sounds crackly with default values

[brlee123]

I currently have a barebones implementation using this package. It seems that even without introducing networking, the voice just doesn't sound crisp (lots of static and sounds 'robotic'). Has anyone else experienced this? I am using just the default values for UniMic and UniVoiceAudioSourceOutput.Factory.

Any help would be greatly appreciated.

[brlee123]

Hi @adrenak, any ideas why this could be happening? As far as I see, I'm using the tool correctly. I'm just using the default values for everything so I expect it should just work. But the audio is almost unintelligible. I really want to get this to work and would appreciate your feedback. Thanks.

[adrenak]

Hi @brlee123 , I'm out of office on a vacation till mid Dec so I'll be unable to look into this much. Having said that, try the following:

• use unimic in an isolated project, no univoice stuff, to test if it is capturing the audio like this. There is probably a scene in unimic that simply plays the audio back to you instantly.
• use univoice with the telepathy network implementation on a local wifi network. Telepathy impl is not perfect yet, but it'll help test if this issue has a network aspect to it
• try making an Android build if you can, build Mono not IL2CPP. univoice-sample repo is build ready. Also, try swapping the network to telepathy in the sample too.
• try changing the source of audiosource-output to directly add the samples to the audio clip (instead of the circular buffer that it currently has). You could make an IOutput impl that's just a very stripped down version of it.
• and ofcourse check your mic :)

I'm hoping I can take out some time and work on the univoice repos, but I likely would not be able to do that

In the meantime I'll try and help you out with any further info you can gather. My discord is adrenak#1934 where you can DM me

[brlee123]

Thanks for the reply @adrenak! I've DM'd you on Discord with my findings so far. Looks like it is indeed the circular buffer that's introducing the "crackling" noise. Hopefully we're able to find an alternative or fix to this.

For anyone else who stumbles upon this, here's what I tried so far- 1) Directly used Unity's Microphone class. Fed data directly into an audio source. This sounded fine. 2) Same as 1 but with UniMic. Sounded fine as well. 3) Used UniVoiceAudioSourceOutput. Created a Factory with default values (buffersegCount = 10, and minSegCount = 5). Created an IAudioOutput with Factory.New() with defaults values (samplingRate = 16000, channelCount = 1, segmentLength = 160). Fed the audio data into IAudioOutput OnSampleReady. This is where the crackling starts.

[brlee123]

@adrenak Another update: I found a way to remove the "crackly" noises by using bufferSegCount = 10, and minSegCount = 2 (all done locally). However, over the network where I've setup a scene with two clients, it seems that only one of the clients is ever able to have clear audio. I am not sure why, but I'm guessing it's because only one of the clients is getting data consistently? I am using my own implementation of UDP. I serialize and queue the audio data as soon as I get them. And on a separate thread, it checks the queue and sends it to the server. This is the same pattern with receiving data on the client. On the server, it is routing to the right recipient as soon as it get these messages.

Everything is being done over LAN so pretty sure this should be okay... but apparently not. Do you think at this point, it could still be a buffer issue, or is it a networking one?

[adrenak]

Hi @brlee123 , did you find a fix for this?

I did some univoice maintenance and the telepathy-network had an issue that kicks in when using one than one client and seems relevant to your issue.

If you have used that repo, or implemented the univoice network interface using that as reference, it might be worth looking into.

As far as values for bufferSegCount and minSegCount are concerned, 10 and 2 seem like better values, but they might change based on network conditions. Looks like an adaptive circular audio buffer would be a good addition.