PCMA/8000 issue stuttering audio back from Google

[JorgMuskens]

Hi Dan, I am running into another problem. When i dial in to the stasis app the invite RTP only show PCMA/8000.

a=rtpmap:8 PCMA/8000 a=rtpmap:96 telephone-event/8000

in asterisk i see the following error when dialin the app

[Mar 19 14:19:40] WARNING[3111]: translate.c:407 framein: no samples for slin 8000khz -> 16000khz

Google responses are stuttering so i think it is not played as 8000khz. Do you have any experience with this?

Regards Jorg

[JorgMuskens]

Additional note is that X-Lite app does not had the issue. Currently I am unable to connect to X-Lite due to a registration error in Asterisk so i am not sure what rtpmap he is negotiating. But my provider is just sending PCMA/8000

[danjenkins]

I'll do a test with my phone limited to to alaw later today for you. But essentially you're getting alaw media in, that should then be transcoded over to signed linear 16 at 16k out and then we send back in signed linear 16 8k which will then be transcoded back to alaw.

if you wanted to cut down on the amount of transcoding etc. you could change your externalMedia channel to output in ulaw (not alaw - https://wiki.asterisk.org/wiki/display/AST/External+Media+and+ARI), change the input format for dialogflow to be AUDIO_ENCODING_MULAW (https://cloud.google.com/dialogflow/docs/reference/rest/v2beta1/QueryInput#audioencoding)

probably wont help your issue at all because it looks like its trying to take what we're sending back (which is 8k) and we're doing doing it quickly enough (or something?) so hence the stuttering? I'd need to try out the scenario myself.

[JorgMuskens]

Switching to AUDIO_ENCODING_MULAW does not work even throws in an error at the end of stream.

{"level":30,"time":1584629585172,"pid":3485,"hostname":"tron-asterisk","name":"Dialogflow-AudioServer","msg":"Got a message sending to dialogflow-asterisk/events","v":1} events.js:292 throw er; // Unhandled 'error' event ^

Error [ERR_STREAM_WRITE_AFTER_END]: write after end at ClientHttp2Stream.Writable.write (_stream_writable.js:286:11) at /usr/src/asterisk-dialogflow-rtp-audioserver/node_modules/@grpc/grpc-js/build/src/call-stream.js:416:34 at processTicksAndRejections (internal/process/task_queues.js:97:5) Emitted 'error' event on Http2CallStream instance at: at errorOrDestroy (internal/streams/destroy.js:128:12) at onwriteError (_stream_writable.js:406:3) at onwrite (_stream_writable.js:427:7) at processTicksAndRejections (internal/process/task_queues.js:83:21) { code: 'ERR_STREAM_WRITE_AFTER_END' } error Command failed with exit code 1. info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

So i switched back. I still thinks it's an Asterisk issue due to the message
WARNING[3597]: translate.c:407 framein: no samples for slin 8000khz -> 16000khz Looks like something is missing but i can find much on google

[danjenkins]

Oh try changing slin16 in the config file to be slin8 (its really slin16 8k I think), change the audio rate from 16000 to 8000 going UP to dialogflow - that would at least mean alaw would be going out to node as slin16 8k, coming back in in slin16 8k and then going back to alaw.

If that doesnt work I'll try and have a look later today (but I'm very busy with work/family because of covid so I'll get to it asap :) )

[JorgMuskens]

That worked a little bit better but still stutter and again throws the error at end of stream The error is a result of not saying anything i think. We need to address this in the code i think. Google throws an error which is not handled by the program. No probs on getting back i will test anyhow

[JorgMuskens]

Looks like it is coming from slicing the audio stream into packets and play it packet by packet. This causes delays which is heard on the phone. I tried to change it that it plays the full audio payload at once but no succes yet. I am still also checking if the Asterisk configuration is correct due to the warning WARNING[5262]: translate.c:407 framein: no samples for slin 8000khz -> 16000khz Not sure if there is an issue with audio codec.

[JorgMuskens]

Hi Dan, most likely the issue with choppy sound could be that we are running asterisk in Hyper-V. So i am going to switch to other machine to ensure that is not an issue. According to some forums i see that there are a lot of problems running in HyperV.

Another thing i found is that according to canonical wave format (http://soundfile.sapp.org/doc/WaveFormat/) each wave file has an offset of 44 bytes. You only slice 42 in the code. You need to change that as this results in the first frame being faulty

On the machine i will also go for version 16 of asterisk but i think 17 works just aswell

[JorgMuskens]

Hi Dan, Hope your doing well these days! I did a lot of testing and came to the conslusion that it is audio encoding issue with the stutter. When i change to ari bridge configuratio RTPserver.format to alaw i get clean audio. The only thing is that the beginning still a small stutter to start, but the welcome message plays for the rest perfectly.

Then i noticed that this change results that audio from my cell is not pickup anymore to send to Google. I will test some more tomorrow.

Also i noticed when 3 channels go in to the bridge the problem is gone with the default settings. This is due to Asterisk is transcoding the incoming data and sending it back to each channel. This is basic behaviour of the ARI bridge. So the issue must be in the encoding and the way we write it back to Asterisk.

I even swapped to physical machine to is massively overpowered for this setup. and switched to Asterisk 16 aswell. So i do not understand what is different from your setup.

Hope we can investigate?\

[danjenkins]

Hey @JorgMuskens I'll try and do some testing on this tonight :) with a alaw phone call

[ghost]

Hi @JorgMuskens - I saw you posted a question about stuttering audio on my issue which is now closed

"@digitaltoast I was wondering if you experience issues on AWS with audio playback from Dialogflow"

I thought I'd reply here as it was related - in short, no! Even on a base "micro" instance, it seems OK for me, but I'm going "pure alaw". But you are using a container aren't you? I dabbled with containers about a year ago - I had audio issues with them so didn't try again.

I just spin up a ubuntu 19.10 minimal, install Asterisk 17.3, and it seems to be OK. With the one weird exception that I seem to need to play a "beep" or indeed anything, to get the audio "flowing" between Asterisk and Dialogflow.

[JorgMuskens]

@digitaltoast ok also in my test with HyperV it looks okay but i have some strange issues with transcoding. For the rest it is working fine. Now i switched back to a older release of Asterisk 16 to see if that make a difference. For the containers no i do not use containers but eventually if the code works i would like to. But i will make my own container as i will need to code to suit my application. Thanks for letting me know!

@danjenkins superb looking forward to your findings

[danjenkins]

@JorgMuskens I just tested sending ulaw out of asterisk, up to dialogflow and then getting 8k slin16 back and it all works flawlessly? Is Is this what you were doing?

[danjenkins]

For ref, I changed the audio input config

audioEncoding: 'AUDIO_ENCODING_MULAW',
sampleRateHertz: 8000,

And changed the ARI project to output ulaw too

format: 'ulaw'

in the config file

That was it, my phone was connected with ulaw, asterisk was sending out ulaw, i was sending in slin16 8k, I got the occasional [2020-03-31 12:08:20] WARNING[31193][C-00000062]: translate.c:407 framein: no samples for lintoulaw in the asterisk console but thats nothing to be too worried about in the grand scheme of things

[danjenkins]

ah the only difference is i negotiated ulaw..... let me run a test with only alaw

[danjenkins]

Just tested with alaw and its fine there too

result of core show channel <channel>

NativeFormats: (alaw)
    WriteFormat: slin
     ReadFormat: alaw
 WriteTranscode: Yes (slin@8000)->(alaw@8000)

[JorgMuskens]

@JorgMuskens I just tested sending ulaw out of asterisk, up to dialogflow and then getting 8k slin16 back and it all works flawlessly? Is Is this what you were doing?

Yes that is the setup i am currently using.

For ref, I changed the audio input config

audioEncoding: 'AUDIO_ENCODING_MULAW',
sampleRateHertz: 8000,

And changed the ARI project to output ulaw too

format: 'ulaw'

in the config file

That was it, my phone was connected with ulaw, asterisk was sending out ulaw, i was sending in slin16 8k, I got the occasional [2020-03-31 12:08:20] WARNING[31193][C-00000062]: translate.c:407 framein: no samples for lintoulaw in the asterisk console but thats nothing to be too worried about in the grand scheme of things

This is were i went wrong forgot to change the audioEncoding: 'AUDIO_ENCODING_MULAW', Will do some more testing.

Just tested with alaw and its fine there too

result of core show channel <channel>

NativeFormats: (alaw)
    WriteFormat: slin
     ReadFormat: alaw
 WriteTranscode: Yes (slin@8000)->(alaw@8000)

That indeed looks promising

[JorgMuskens]

The change and new code work much better. I only have a little very small stutter in the begin of each speech element that is comming from google. It looks like it needs to fire up something and then eventually its clean audio like an old desil engine. Perhaps not the best way to explain this.

I will ignore the warning on the samples.

Perhaps MQTT is not fast enough on opening the channel so a small delay is experienced but this would not explain the first word stutter on each sentence coming from google back when the channel is open. So the problem shifts from complete stutter to only first word or couple of letters stutter.

Perhaps we need to close this as i think main issue is resolved with new code setup.

Superb support 👍

[danjenkins]

Great! I changed to code to remove 44 bytes.... maybe I should only be removing 42? Try moving back to 42 :) Closing :)

[JorgMuskens]

Hmmz did that and it looks like its working. Strange stuff!