streamingdetectintent not working

[sahis]

Problem : It detects the words I say and brings the result. However, the voice detection is always on, even if I turn off the mic, the voice recognition continues and it derives words by itself. and the query never ends. When I connect with Zoiper and call, there is no problem, everything works correctly. This problem is encountered because a phone is called with sip (voip). ------log------- 3|aiagent_rtpaudioserver | {"level":30,"time":1607373924246,"pid":13812,"hostname":"pbxtest.gitasis.com","name":"Dialogflow-AudioServer","id":"o-100002-8-4821","transcript":"Selam merhaba Cumhur Cumhur Cumhur","msg":"Intermediate transcript","v":1} 3|aiagent_rtpaudioserver | {"level":30,"time":1607373924246,"pid":13812,"hostname":"pbxtest.gitasis.com","name":"Dialogflow-AudioServer","id":"o-100002-8-4821","body":{"transcript":{"speechWordInfo":[],"messageType":"TRANSCRIPT","transcript":"Selam merhaba Cumhur Cumhur Cumhur","isFinal":false,"confidence":0,"stability":0.8999999761581421,"speechEndOffset":{"seconds":"46","nanos":260000000}},"intent":null},"msg":"Dialogflow data","v":1} 3|aiagent_rtpaudioserver | {"level":30,"time":1607373924246,"pid":13812,"hostname":"pbxtest.gitasis.com","name":"Dialogflow-AudioServer","msg":"Got a message sending to dialogflow-asterisk/o-100002-8-4821/events","v":1} 2|aiagent_aribridge | {"level":30,"time":1607373924342,"pid":13801,"hostname":"pbxtest.gitasis.com","name":"Asterisk-Dialogflow-ARI-Bridge","topic":"dialogflow-asterisk/o-100002-8-4821/events","payload":{"transcript":{"speechWordInfo":[],"messageType":"TRANSCRIPT","transcript":"Selam merhaba Cumhur Cumhur Cumhur","isFinal":false,"confidence":0,"stability":0.8999999761581421,"speechEndOffset":{"seconds":"46","nanos":260000000}},"intent":null},"msg":"got a message","v":1} 2|aiagent_aribridge | debug: null 2|aiagent_aribridge | {"level":30,"time":1607373925813,"pid":13801,"hostname":"pbxtest.gitasis.com","name":"Asterisk-Dialogflow-ARI-Bridge","type":"ChannelLeftBridge","timestamp":"2020-12-07T21:45:25.811+0100","bridge":{"id":"ddc82adb-ad32-4222-bd27-72a7de0e5e0e","technology":"simple_bridge","bridge_type":"mixing","bridge_class":"stasis","creator":"Stasis","name":"","channels":["49fb76a0-0a1c-4fd9-8ae4-0b8f57e60266"],"creationtime":"2020-12-07T21:44:36.717+0100","video_mode":"talker"},"channel":{"id":"o-100002-8-4821","name":"Local/DIAL@ar_ai-00000065;1","state":"Up","caller":{"name":"","number":"DIAL"},"connected":{"name":"","number":""},"accountcode":"11187","dialplan":{"context":"ar_ai","exten":"ANSWERED","priority":18,"app_name":"Stasis","app_data":"dialogflow"},"creationtime":"2020-12-07T21:44:26.691+0100","language":"en"},"asterisk_id":"00:0c:29:53:c6:4b","application":"dialogflow","msg":"Channel left bridge","v":1}

Not:"transcript":"Selam merhaba Cumhur Cumhur Cumhur" "Cumhur Cumhur Cumhur" I did not say

---confs

rtpServer: { host: '127.0.0.1', port: '7777', //format: 'slin16' //tried the result is the same format: 'ulaw' },

audioInputConfig: { //audioEncoding: 'AUDIO_ENCODING_LINEAR_16', // tried the result is the same audioEncoding: 'AUDIO_ENCODING_MULAW', //sampleRateHertz: 16000, //tried the result is the same sampleRateHertz: 8000, languageCode: 'tr', singleUtterance: false },

Thank you very much in advance.

[danjenkins]

Hi - so even if you turn off your mic, Asterisk is still sending blank audio over to this process. We could potentially look at the audio frames coming in and see if they are actually blank and therefore not send them up to the provider.

Does this help? Sorry, I've been busy with work and haven't had the time to look at github issues.

[sahis]

hello, thank you very much for your answer. So what exactly is the solution to this? What kind of path should we follow?

[danjenkins]

How do you mute? Or the phone itself? I'm pretty busy with client work at the moment so picking anything up is near impossible until end of Feb at the earliest.

[sahis]

actually there is a problem like this. No problem when calling fixed lines, but not detecting that the customer is silent when calling mobile dialogflow

[Toni-Navarro]

actually there is a problem like this. No problem when calling fixed lines, but not detecting that the customer is silent when calling mobile dialogflow

Hi sahis,

I was experimenting the same issue when I called from a landline (via sip trunk). I tried to change singleUtterance to true in config. No sure why, first time it failed and returned an error in audioserver but second time worked properly. Could you try it?

[sahis]

thank you answer hello, i made the setting you specified (singleUtterance : true). However, I got the error as below. Can you please help?

debug e: { fulfillmentMessages: [], 4|aiagent_aribridge | outputContexts: [], 4|aiagent_aribridge | queryText: '', 4|aiagent_aribridge | speechRecognitionConfidence: 0, 4|aiagent_aribridge | action: '', 4|aiagent_aribridge | parameters: null, 4|aiagent_aribridge | allRequiredParamsPresent: false, 4|aiagent_aribridge | fulfillmentText: '', 4|aiagent_aribridge | webhookSource: '', 4|aiagent_aribridge | webhookPayload: null, 4|aiagent_aribridge | intent: null, 4|aiagent_aribridge | intentDetectionConfidence: 0, 4|aiagent_aribridge | diagnosticInfo: null, 4|aiagent_aribridge | languageCode: 'de-DE', 4|aiagent_aribridge | sentimentAnalysisResult: null, 4|aiagent_aribridge | knowledgeAnswers: null } TypeError: Cannot read property 'fields' of null 4|aiagent_aribridge | at Bridge.bridge.on (/etc/aiagent/ari-bridge/index.js:88:49) 4|aiagent_aribridge | at emitOne (events.js:116:13) 4|aiagent_aribridge | at Bridge.emit (events.js:211:7) 4|aiagent_aribridge | at Bridge.receivedDialogFlowEvent (/etc/aiagent/ari-bridge/lib/Bridge.js:31:14) 4|aiagent_aribridge | at MqttClient.mqttClient.on (/etc/aiagent/ari-bridge/index.js:29:28) 4|aiagent_aribridge | at emitThree (events.js:136:13) 4|aiagent_aribridge | at MqttClient.emit (events.js:217:7) 4|aiagent_aribridge | at MqttClient._handlePublish (/etc/aiagent/ari-bridge/node_modules/async-mqtt/node_modules/mqtt/lib/client.js:1277:12) 4|aiagent_aribridge | at MqttClient._handlePacket (/etc/aiagent/ari-bridge/node_modules/async-mqtt/node_modules/mqtt/lib/client.js:410:12) 4|aiagent_aribridge | at work (/etc/aiagent/ari-bridge/node_modules/async-mqtt/node_modules/mqtt/lib/client.js:321:12) 4|aiagent_aribridge | at Writable.writable._write (/etc/aiagent/ari-bridge/node_modules/async-mqtt/node_modules/mqtt/lib/client.js:335:5) 4|aiagent_aribridge | at doWrite (/etc/aiagent/ari-bridge/node_modules/readable-stream/lib/_stream_writable.js:409:139) 4|aiagent_aribridge | at writeOrBuffer (/etc/aiagent/ari-bridge/node_modules/readable-stream/lib/_stream_writable.js:398:5) 4|aiagent_aribridge | at Writable.write (/etc/aiagent/ari-bridge/node_modules/readable-stream/lib/_stream_writable.js:307:11) 4|aiagent_aribridge | at Socket.ondata (_stream_readable.js:639:20) 4|aiagent_aribridge | at emitOne (events.js:116:13)

[evaldodeoliveira]

https://github.com/nimbleape/asterisk-dialogflow-rtp-audioserver/issues/20#issuecomment-783266239

The same thing happened to me.

[iagoccampos]

Did someone figured that out?

Calls via sip trunk does not trigger an intent. Changing singleUtterance option to true seems to be a good option, but it throws an erro "write after end". It's looks like the event "WELCOME" and audio recognition starts at the same time. If the welcome audio is big the stream is closed before the audio finishes, throwing the error.