MRCPRecog unimrcp + pocketshinx No-Input-Timerout does not work

[achaloyan]

Originally reported on Google Code with ID 156

What steps will reproduce the problem?
1. Use MRCPRecog and PocketSphinx in Asterisk setting b=0, and nit
2. Watch output from mrcpserver and you will see the nit in the Recognize command.
3. The mrcp server never processes the nit.

What version of the product are you using? On what operating system?
Latest version of unimrcp on CentOS 6.3

Please provide any additional information below.

Running this MRCPRecog command as follows:
MRCPRecog(/usr/local/unimrcp/data/posneg.jsgf,p=default&t=5000&b=0&nit=5000&ct=0.7&spl=en-US&sit=true&f=prompt)

You can see the Rccognize request with the nit set:

2013-04-13 12:04:42:637295 [INFO]   Receive MRCPv2 Stream 10.220.1.26:1544 <-> 10.220.1.26:47652
[297 bytes]
MRCP/2.0 297 RECOGNIZE 2
Channel-Identifier: 39e53058a45c11e2@speechrecog
Content-Type: text/uri-list
Recognition-Mode: false
Recognition-Timeout: 5000
Confidence-Threshold: 0.7
Speech-Language: en-US
Start-Input-Timers: true
No-Input-Timeout: 5000
Content-Length: 17

session:grammar-0
2013-04-13 12:04:42:637387 [INFO]   Process RECOGNIZE Request <39e53058a45c11e2@speechrecog>
[2]
2013-04-13 12:04:42:637422 [INFO]   Dispatch Request RECOGNIZE <39e53058a45c11e2@pocketsphinx>
2013-04-13 12:04:42:637489 [INFO]   Open Waveform File [/tmp/utter-39e53058a45c11e2-2.pcm]
<39e53058a45c11e2@pocketsphinx>
2013-04-13 12:04:42:637575 [INFO]   Process RECOGNIZE Response <39e53058a45c11e2@speechrecog>
[2]
2013-04-13 12:04:42:637586 [INFO]   State Transition IDLE -> RECOGNIZING <39e53058a45c11e2@speechrecog>
2013-04-13 12:04:42:637746 [INFO]   Send MRCPv2 Stream 10.220.1.26:1544 <-> 10.220.1.26:47652
[83 bytes]
MRCP/2.0 83 2 200 IN-PROGRESS
Channel-Identifier: 39e53058a45c11e2@speechrecog

But if there is no audio input, the MRCP Server never times out.  Am I correct in assuming
that if the nit is set, the server will at some point timeout, or rather, pocketsphinx
would send an nit message to the server saying to discontinue?

Also, I found the nit setting in pocketsphinx.xml in the conf directory, it does nothing.

Would it be better to have asterisk handle the nit instead of pocketsphinx?  Perhaps
pocketsphinx in incapable of performing an nit.

Can someone point me in the correct direction here please..

Also, one quick unrelated question, I see that the client/server ports are fixed, does
this mean that unimrcp can only handle one call at a time, or can it handle multiple,
thx.

Reported by damiank244 on 2013-04-13 17:10:46

[achaloyan]

Additional info:

in mrcp_pocketsphinx.c this statement is always false:
if(recognizer->is_input_timer_on == TRUE) 

Reported by damiank244 on 2013-04-13 17:19:02

[achaloyan]

the recognizer->is_input_timer_on flag is always false, but, even if it were set to
true at the beginning of the call...

Theres another problem here.  If the microphone is completed muted before placing the
call, this statement is executed once or twice anyways:
if(score != 0 && recognizer->is_input_timer_on)
{
       recognizer->is_input_timer_on = FALSE;
}

Not sure what can be done here....  There are partial results even though the call
is completely muted before placing.  

Reported by damiank244 on 2013-04-13 17:27:47

[achaloyan]

Fixed....
Commenting the following:
/* reset input timer as we have partial match now */
//              if(score != 0 && recognizer->is_input_timer_on)
        //      {
//                    recognizer->is_input_timer_on = FALSE;
//              }

And adding this:

det_event = mpf_activity_detector_process(recognizer->detector,frame);
  switch(det_event) {
      case MPF_DETECTOR_EVENT_ACTIVITY:
++           if(recognizer->is_input_timer_on)
++           {
++              apt_log(APT_LOG_MARK,APT_PRIO_INFO,"Stopping input timers..... "APT_SIDRES_FMT
,RECOGNIZER_SIDRES(recognizer));
++                recognizer->is_input_timer_on = FALSE;
++           }
           apt_log(APT_LOG_MARK,APT_PRIO_INFO,"Detected Voice Activity "APT_SIDRES_FMT,RECOGNIZER_SIDRES(recognizer));
                    pocketsphinx_start_of_input(recognizer);
                break;
     case MPF_DETECTOR_EVENT_INACTIVITY:
      apt_log(APT_LOG_MARK,APT_PRIO_INFO,"Detected Voice Inactivity "APT_SIDRES_FMT,
                                      RECOGNIZER_SIDRES(recognizer));
                                pocketsphinx_end_of_input(recognizer,RECOGNIZER_COMPLETION_CAUSE_SUCCESS);
                                break;
                        default:
                                break;
                }

For some reason, the pocketsphinx plugin gets partial results even if the microphone
is muted, however, even though pocketsphinx is getting partial results, MRCP correctly
detects voice activity, therefore, stopping the input timers where voice is properly
detected fixes the issue.

A patch is attached..

Reported by damiank244 on 2013-04-13 17:57:57

---

• Attachment: mrcppatch

[achaloyan]

Reported by achaloyan on 2013-04-16 01:45:18

• Labels added: Type-Defect, Priority-Medium, OpSys-All, Component-PocketSphinx

[achaloyan]

This issue relates to the PocketSphinx plugin which is no longer supported.