Fail to compile mod_openai_asr

[hnimminh]

Fail to install mod_openai_asr with FreeSWITCH, below is an error.

@akscf Do you have any installation guide for this FreeSWITCH module? is there any dependency required?

#0 164.0 making all mod_openai_asr                                                                                                                                                                                                                                                                                          
#0 164.0 make[4]: Entering directory '/usr/src/freeswitch/src/mod/applications/mod_openai_asr'                                                                                                                                                                                                                              
#0 164.0 /usr/src/freeswitch/src/mod/applications/mod_openai_asr                                                                                                                                                                                                                                                            
#0 164.0 make[5]: Entering directory '/usr/src/freeswitch/src/mod/applications/mod_openai_asr'                                                                                                                                                                                                                              
#0 164.0 Compiling /usr/src/freeswitch/src/mod/applications/mod_openai_asr/mod_openai_asr.c...                                                                                                                                                                                                                              
#0 164.0 libtool: compile:  gcc -I/usr/include/uuid -I/usr/src/freeswitch/src/include -I/usr/src/freeswitch/src/include -I/usr/src/freeswitch/libs/libteletone/src -fPIC -ffast-math -Werror -Wno-unused-result -Wno-misleading-indentation -fvisibility=hidden -DSWITCH_API_VISIBILITY=1 -DCJSON_API_VISIBILITY=1 -DHAVE_VI
SIBILITY=1 -g -ggdb -DHAVE_OPENSSL -g -O2 -Wall -std=c99 -pedantic -Wdeclaration-after-statement -D_GNU_SOURCE -DHAVE_CONFIG_H -c /usr/src/freeswitch/src/mod/applications/mod_openai_asr/mod_openai_asr.c  -fPIC -DPIC -o .libs/mod_openai_asr.o                                                                           
#0 164.2   CCLD     mod_native_file.la                                                                                                                                                                                                                                                                                      
#0 164.2 /usr/src/freeswitch/src/mod/applications/mod_openai_asr/mod_openai_asr.c: In function 'asr_feed':                                                                                                                                                                                                                  
#0 164.2 /usr/src/freeswitch/src/mod/applications/mod_openai_asr/mod_openai_asr.c:382:56: error: pointer of type 'void *' used in arithmetic [-Werror=pointer-arith]                                                                                                                                                        
#0 164.2                      memcpy(tau_buf->data, (void *)(ptr + hdr_ofs), hdr_sz);                                                                                                                                                                                                                                       
#0 164.2                                                         ^                                                                                                                                                                                                                                                          
#0 164.2 /usr/src/freeswitch/src/mod/applications/mod_openai_asr/mod_openai_asr.c:383:66: error: pointer of type 'void *' used in arithmetic [-Werror=pointer-arith]                                                                                                                                                        
#0 164.2                      memcpy(tau_buf->data + hdr_sz , (void *)(ptr + 0), vblen);
#0 164.2                                                                   ^
#0 164.2 /usr/src/freeswitch/src/mod/applications/mod_openai_asr/mod_openai_asr.c:398:56: error: pointer of type 'void *' used in arithmetic [-Werror=pointer-arith]
#0 164.2                      memcpy(tau_buf->data, (void *)(ptr + ofs), rlen); 
#0 164.2                                                         ^
#0 164.4 make[4]: Leaving directory '/usr/src/freeswitch/src/mod/formats/mod_native_file'
#0 164.4   CXX      libluamod_la-mod_lua_wrap.lo
#0 164.4 cc1: all warnings being treated as errors
#0 164.4 make[5]: Leaving directory '/usr/src/freeswitch/src/mod/applications/mod_openai_asr'
#0 164.4 make[5]: *** [../../../../build/modmake.rules:216: mod_openai_asr.lo] Error 1
#0 164.4 make[4]: *** [/usr/src/freeswitch/build/modmake.rules:89: all] Error 1 
#0 164.4 make[4]: Leaving directory '/usr/src/freeswitch/src/mod/applications/mod_openai_asr'
#0 164.4 make[3]: *** [Makefile:728: mod_openai_asr-all] Error 1
#0 164.4 make[3]: *** Waiting for unfinished jobs....

[akscf]

Hi,

is there any dependency required? No, there is no any dependencies.

Do you have any installation guide for this FreeSWITCH module? Also not, but just follow a standard Freeswitch guides for building modules, it should work.

error: pointer of type 'void *' used in arithmetic [-Werror=pointer-arith] this flag already present in the Makefile.am,
check your Makefile, seems it was generated with issues.

Try to remove: Makefile and Makefile.in from this module, and perform: make clean and automake (from the Freeswitch sources root path) This should recreate Makefile

[hnimminh]

Hi @akscf thanks for your answer. I tried but issue still same, or may be I missed a step or I did it wrong (I don't know). Could you please describe step by step. It would help much.

[akscf]

try to create a new installation:

after unpack the Freeswitch sources and performing ./bootstrap.sh put this module into 'src/mod/asr_tts/' and add 'asr_tts/mod_openai_asr' into modules.conf after that perform: ./configure

also make sure that the module appeared in: configure.ac

and perform: make clean install

[hnimminh]

I followed exactly what you describe but, module does not appear in configure.ac and build still failed.

cause by module missing Makefile Makefile.am like other module? I doubt if module can work.

[akscf]

got it, try this

1) make clean

2) open configure.ac and add: src/mod/asr_tts/mod_openai_asr/Makefile in section AC_CONFIG_FILES just before a first module

3) perform from the freeswitch roor dir: automake && make clean it should make Makefile and Makefile.in for the module or throw some errors

[hnimminh]

yes, finally. It was build successfully. can you tell me how to get there result? listen on event or get from channel variable?

[akscf]

nice, yes there is a chanvar: detect_speech_result

 <action application="log" data="CRIT SPEECH_RESULT=${detect_speech_result}"/>

[hnimminh]

Nice, It work. I discover it transcribe input audio to text (with openai) during play audio.

I wonder if:

Is this module able to continuously transcribe audio? and run in backround? example: background_detect_speech(detect:openai {lang=en,duration=60,chunk=5}) it mean:

• module just silent and listen input audio in 60s (by duration=60)
• and send audio to openai every 5s (by chunk=5)

similar to mod_audio_stream but apply for openai (instead of IBM Watson)

[akscf]

I discover it transcribe input audio to text (with openai) during play audio.

btw, seems it's a something bug in Freeswitch, if you capture audio after playback there are noises there (look likes wrong transcoded pcm)

Is this module able to continuously transcribe audio? and run in backround?

actually it's already implemented some another product, mb I'll find time to transfer it here...

but you can look at this: https://github.com/akscf/mod_quickjs and example: https://github.com/akscf/mod_quickjs/blob/main/examples/v16_chatgpt.js

it's a little similar that you described