search

vavrek / Open-Assistant-Version-Zero

Open Source Voice Assistant
https://openassistant.org
GNU General Public License v3.0
318 stars 70 forks source link

Segmentation Fault #13

Open S1MB10T3 opened 6 years ago

S1MB10T3 commented 6 years ago

After running ./run.sh -c -H20 -m0 -M mind/boot on both my Debian laptop and my raspberry pi I get a segmentation fault.

Console Log:

pi@raspberrypi:~/openassistant$ ./run.sh -c -H20 -m0 -M mind/boot                                                                                                 
Current configuration:
[NAME]                  [DEFLT]         [VALUE]
-agc                    none            none
-agcthresh              2.0             2.000000e+00
-allphone
-allphone_ci            no              no
-alpha                  0.97            9.700000e-01
-ascale                 20.0            2.000000e+01
-aw                     1               1
-backtrace              no              no
-beam                   1e-48           1.000000e-48
-bestpath               yes             yes
-bestpathlw             9.5             9.500000e+00
-ceplen                 13              13
-cmn                    current         current
-cmninit                8.0             8.0
-compallsen             no              no
-debug                                  0
-dict
-dictcase               no              no
-dither                 no              no
-doublebw               no              no
-ds                     1               1
-fdict
-feat                   1s_c_d_dd       1s_c_d_dd
-featparams
-fillprob               1e-8            1.000000e-08
-frate                  100             100
-fsg
-fsgusealtpron          yes             yes
-fsgusefiller           yes             yes
-fwdflat                yes             yes
-fwdflatbeam            1e-64           1.000000e-64
-fwdflatefwid           4               4
-fwdflatlw              8.5             8.500000e+00
-fwdflatsfwin           25              25
-fwdflatwbeam           7e-29           7.000000e-29
-fwdtree                yes             yes
-hmm
-input_endian           little          little
-jsgf
-keyphrase
-kws
-kws_delay              10              10
-kws_plp                1e-1            1.000000e-01
-kws_threshold          1               1.000000e+00
-latsize                5000            5000
-lda
-ldadim                 0               0
-lifter                 0               0
-lm
-lmctl
-lmname
-logbase                1.0001          1.000100e+00
-logfn
-logspec                no              no
-lowerf                 133.33334       1.333333e+02
-lpbeam                 1e-40           1.000000e-40
-lponlybeam             7e-29           7.000000e-29
-lw                     6.5             6.500000e+00
-maxhmmpf               30000           30000
-maxwpf                 -1              -1
-mdef
-mean
-mfclogdir
-min_endfr              0               0
-mixw
-mixwfloor              0.0000001       1.000000e-07
-mllr
-mmap                   yes             yes
-ncep                   13              13
-nfft                   512             512
-nfilt                  40              40
-nwpen                  1.0             1.000000e+00
-pbeam                  1e-48           1.000000e-48
-pip                    1.0             1.000000e+00
-pl_beam                1e-10           1.000000e-10
-pl_pbeam               1e-10           1.000000e-10
-pl_pip                 1.0             1.000000e+00
-pl_weight              3.0             3.000000e+00
-pl_window              5               5
-rawlogdir
-remove_dc              no              no
-remove_noise           yes             yes
-remove_silence         yes             yes
-round_filters          yes             yes
-samprate               16000           1.600000e+04
-seed                   -1              -1
-sendump
-senlogdir
-senmgau
-silprob                0.005           5.000000e-03
-smoothspec             no              no
-svspec
-tmat
-tmatfloor              0.0001          1.000000e-04
-topn                   4               4
-topn_beam              0               0
-toprule
-transform              legacy          legacy
-unit_area              yes             yes
-upperf                 6855.4976       6.855498e+03
-uw                     1.0             1.000000e+00
-vad_postspeech         50              50
-vad_prespeech          20              20
-vad_startspeech        10              10
-vad_threshold          2.0             2.000000e+00
-var
-varfloor               0.0001          1.000000e-04
-varnorm                no              no
-verbose                no              no
-warp_params
-warp_type              inverse_linear  inverse_linear
-wbeam                  7e-29           7.000000e-29
-wip                    0.65            6.500000e-01
-wlen                   0.025625        2.562500e-02

Segmentation fault
joshuashort commented 6 years ago

Try running with a different logging level (e.g. logging.DEBUG) https://github.com/vavrek/openassistant/blob/6a51f51c0599d4843310334adffe3a4a6f3404e8/run.py#L6 I wish there was a switch to do that.

And I haven't poked around in the code recently enough to know what phase of running you're getting to. That output is from Sphinx.. is -m0 the correct audio for your system? Try leaving that parameter out of the command when you run it (that should make it try to figure out the system default device).

Dennsen commented 6 years ago

I have the same problem. Here's the output with DEBUG enabled. It's a Ubuntu 16.04 system.

$./run.sh -c -H20 -M mind/boot DEBUG:modules.speech_recognition.gst:Loading DEBUG:main:Arguments: Namespace(continuous=True, history=20, invalid_sentence_command=None, microphone=None, mind_dir='mind/boot', pass_words=False, valid_sentence_command=None) INFO:core.util.config:Loading Mind: mind/boot DEBUG:main:Configuring Module: Language DEBUG:modules.language:Updating Language INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): www.speech.cs.cmu.edu DEBUG:requests.packages.urllib3.connectionpool:"POST /cgi-bin/tools/lmtool/run HTTP/1.1" 302 0 DEBUG:requests.packages.urllib3.connectionpool:"GET /tools/product/1520770653_15884 HTTP/1.1" 301 273 DEBUG:requests.packages.urllib3.connectionpool:"GET /tools/product/1520770653_15884/ HTTP/1.1" 200 None INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): www.speech.cs.cmu.edu DEBUG:requests.packages.urllib3.connectionpool:"GET /tools/product/1520770653_15884/4137.lm HTTP/1.1" 200 995 INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): www.speech.cs.cmu.edu DEBUG:requests.packages.urllib3.connectionpool:"GET /tools/product/1520770653_15884/4137.dic HTTP/1.1" 200 93 DEBUG:main:Configuring Module: Speech Recognition DEBUG:modules.speech_recognition.gst:Initializing Recognizer DEBUG:modules.speech_recognition.gst:<core.util.config.Config object at 0x7f4e51d3feb8> DEBUG:modules.speech_recognition.gst:{'invalid_sentence_command': False, 'continuous': False, 'valid_sentence_command': False, 'interface': None, 'pass_words': False, 'microphone': True, 'history': True} DEBUG:modules.speech_recognition.gst:autoaudiosrc device="hw:True,0" ! audioconvert ! audioresample ! pocketsphinx lm=mind/boot/cache/lm dict=mind/boot/cache/dic ! appsink sync=false Current configuration: [NAME] [DEFLT] [VALUE] -agc none none -agcthresh 2.0 2.000000e+00 -allphone
-allphone_ci no no -alpha 0.97 9.700000e-01 -ascale 20.0 2.000000e+01 -aw 1 1 -backtrace no no -beam 1e-48 1.000000e-48 -bestpath yes yes -bestpathlw 9.5 9.500000e+00 -ceplen 13 13 -cmn current current -cmninit 8.0 8.0 -compallsen no no -debug 0 -dict
-dictcase no no -dither no no -doublebw no no -ds 1 1 -fdict
-feat 1s_c_d_dd 1s_c_d_dd -featparams
-fillprob 1e-8 1.000000e-08 -frate 100 100 -fsg
-fsgusealtpron yes yes -fsgusefiller yes yes -fwdflat yes yes -fwdflatbeam 1e-64 1.000000e-64 -fwdflatefwid 4 4 -fwdflatlw 8.5 8.500000e+00 -fwdflatsfwin 25 25 -fwdflatwbeam 7e-29 7.000000e-29 -fwdtree yes yes -hmm
-input_endian little little -jsgf
-keyphrase
-kws
-kws_delay 10 10 -kws_plp 1e-1 1.000000e-01 -kws_threshold 1 1.000000e+00 -latsize 5000 5000 -lda
-ldadim 0 0 -lifter 0 0 -lm
-lmctl
-lmname
-logbase 1.0001 1.000100e+00 -logfn
-logspec no no -lowerf 133.33334 1.333333e+02 -lpbeam 1e-40 1.000000e-40 -lponlybeam 7e-29 7.000000e-29 -lw 6.5 6.500000e+00 -maxhmmpf 30000 30000 -maxwpf -1 -1 -mdef
-mean
-mfclogdir
-min_endfr 0 0 -mixw
-mixwfloor 0.0000001 1.000000e-07 -mllr
-mmap yes yes -ncep 13 13 -nfft 512 512 -nfilt 40 40 -nwpen 1.0 1.000000e+00 -pbeam 1e-48 1.000000e-48 -pip 1.0 1.000000e+00 -pl_beam 1e-10 1.000000e-10 -pl_pbeam 1e-10 1.000000e-10 -pl_pip 1.0 1.000000e+00 -pl_weight 3.0 3.000000e+00 -pl_window 5 5 -rawlogdir
-remove_dc no no -remove_noise yes yes -remove_silence yes yes -round_filters yes yes -samprate 16000 1.600000e+04 -seed -1 -1 -sendump
-senlogdir
-senmgau
-silprob 0.005 5.000000e-03 -smoothspec no no -svspec
-tmat
-tmatfloor 0.0001 1.000000e-04 -topn 4 4 -topn_beam 0 0 -toprule
-transform legacy legacy -unit_area yes yes -upperf 6855.4976 6.855498e+03 -uw 1.0 1.000000e+00 -vad_postspeech 50 50 -vad_prespeech 20 20 -vad_startspeech 10 10 -vad_threshold 2.0 2.000000e+00 -var
-varfloor 0.0001 1.000000e-04 -varnorm no no -verbose no no -warp_params
-warp_type inverse_linear inverse_linear -wbeam 7e-29 7.000000e-29 -wip 0.65 6.500000e-01 -wlen 0.025625 2.562500e-02

INFO:core.assistant:Initializing Assistant DEBUG:core.assistant:<core.util.config.Config object at 0x7f4e51d3feb8> Segmentation fault

small update: dmsg shows

autoaudiosrc0-a[7255]: segfault at 10 ip 00007f09a12c23da sp 00007f09977fd810 error 4 in libpocketsphinx.so.3.0.0[7f09a128c000+45000]

update nr. 2: running the pipeline manually also leads to a segfault:

gst-launch-1.0 -vvv autoaudiosrc ! audioconvert ! audioresample ! pocketsphinx lm=mind/boot/cache/lm dict=mind/boot/cache/dic ! appsink sync=false ... Leitung wird auf PAUSIERT gesetzt ... Leitung ist aktiv und erfordert keinen VORLAUF … Leitung wird auf ABSPIELEN gesetzt ... New clock: GstPulseSrcClock /GstPipeline:pipeline0/GstAutoAudioSrc:autoaudiosrc0/GstPulseSrc:autoaudiosrc0-actual-src-puls: source-output-index = 24 /GstPipeline:pipeline0/GstAutoAudioSrc:autoaudiosrc0/GstPulseSrc:autoaudiosrc0-actual-src-puls: actual-buffer-time = 200000 /GstPipeline:pipeline0/GstAutoAudioSrc:autoaudiosrc0/GstPulseSrc:autoaudiosrc0-actual-src-puls: actual-latency-time = 10000 Verzögerung neu verteilen … /GstPipeline:pipeline0/GstAutoAudioSrc:autoaudiosrc0/GstPulseSrc:autoaudiosrc0-actual-src-puls.GstPad:src: caps = "audio/x-raw\,\ format\=(string)S16LE\,\ layout\=(string)interleaved\,\ rate\=(int)16000\,\ channels\=(int)1" /GstPipeline:pipeline0/GstAutoAudioSrc:autoaudiosrc0.GstGhostPad:src: caps = "audio/x-raw\,\ format\=(string)S16LE\,\ layout\=(string)interleaved\,\ rate\=(int)16000\,\ channels\=(int)1" /GstPipeline:pipeline0/GstAudioConvert:audioconvert0.GstPad:src: caps = "audio/x-raw\,\ format\=(string)S16LE\,\ layout\=(string)interleaved\,\ rate\=(int)16000\,\ channels\=(int)1" /GstPipeline:pipeline0/GstAudioResample:audioresample0.GstPad:src: caps = "audio/x-raw\,\ format\=(string)S16LE\,\ layout\=(string)interleaved\,\ rate\=(int)16000\,\ channels\=(int)1" /GstPipeline:pipeline0/GstPocketSphinx:pocketsphinx0.GstPad:sink: caps = "audio/x-raw\,\ format\=(string)S16LE\,\ layout\=(string)interleaved\,\ rate\=(int)16000\,\ channels\=(int)1" /GstPipeline:pipeline0/GstAudioResample:audioresample0.GstPad:sink: caps = "audio/x-raw\,\ format\=(string)S16LE\,\ layout\=(string)interleaved\,\ rate\=(int)16000\,\ channels\=(int)1" /GstPipeline:pipeline0/GstAudioConvert:audioconvert0.GstPad:sink: caps = "audio/x-raw\,\ format\=(string)S16LE\,\ layout\=(string)interleaved\,\ rate\=(int)16000\,\ channels\=(int)1" /GstPipeline:pipeline0/GstAutoAudioSrc:autoaudiosrc0.GstGhostPad:src.GstProxyPad:proxypad0: caps = "audio/x-raw\,\ format\=(string)S16LE\,\ layout\=(string)interleaved\,\ rate\=(int)16000\,\ channels\=(int)1" /GstPipeline:pipeline0/GstAutoAudioSrc:autoaudiosrc0/GstPulseSrc:autoaudiosrc0-actual-src-puls: volume = 1 /GstPipeline:pipeline0/GstAutoAudioSrc:autoaudiosrc0/GstPulseSrc:autoaudiosrc0-actual-src-puls: mute = false /GstPipeline:pipeline0/GstAutoAudioSrc:autoaudiosrc0/GstPulseSrc:autoaudiosrc0-actual-src-puls: current-device = alsa_input.usb-Kingston_HyperX_7.1_Audio_00000000-00.analog-stereo Caught SIGSEGV Spinning. Please run 'gdb gst-launch-1.0 10849' to continue debugging, Ctrl-C to quit, or Ctrl-\ to dump core.

joshuashort commented 6 years ago

Thanks @Dennsen! I started dusting off my old development branch.. hope to update some stuff and move to the 'official' OA repository soonish (https://github.com/openassistant) -- I've got a branch that includes -d for executing with debug-level logging. But that's a separate topic..

The first issue I noticed was a line in your debug output where it's printing the pipeline, and it's using True as the device number.. this comes from an incorrect default configuration at https://github.com/vavrek/openassistant/blob/6a51f51c0599d4843310334adffe3a4a6f3404e8/mind/boot/conf/settings.json#L4 -- it shouldn't be true, it should actually be a number representing the audio device.. like 0 or 1. It doesn't look like this version of the code respect run-time arguments when there are defaults set, so try updating that configuration file before running. If your audio device is 0, this block (https://github.com/vavrek/openassistant/blob/6a51f51c0599d4843310334adffe3a4a6f3404e8/modules/speech_recognition/gst.py#L26-L31) doesn't actually do the right thing.. it should be changed to if src is not None -- 0 is a valid device number, but it'll be treated as False for this initialization logic.

I guess I'm most concerned about what's happening when you try to manually run the pipeline.. how recent is your gst/pocketsphinx install? How did you install pocketsphinx? Do you also have the other gst-plugins (i.e. good, bad, ugly)?

Dennsen commented 6 years ago

Hey, thanks for the answer.

I installed pocketsphinx via: sudo apt install gstreamer1.0-pocketsphinx All other gst-plugins were also installed. After installing a new version of pocketsphinx like described here in the link: https://bakerstreetsystems.com/blog/post/setting-cmu-sphinx-pocketsphinx-ubuntu-1404 it works without a segfault. Don't know, what's wrong with the default lib.

Guess I can live with that for now, but we should keep an eye on this.

joshuashort commented 6 years ago

Good call! I haven't had much luck with the package installs of pocketsphinx.. I think our advice for now is to simply build from source.

Just one more thing, @Dennsen, does the output from pocketsphinx now show the expected paths for lm, dict, and hmm?

Dennsen commented 6 years ago

Good point, I checked the output from the following pipe, seems like the wrong path is used:

gst-launch-1.0 -vvv autoaudiosrc ! audioconvert ! audioresample ! pocketsphinx lm=mind/boot/cache/lm dict=mind/boot/cache/dic ! appsink sync=false .... -dict /usr/local/share/pocketsphinx/model/en-us/cmudict-en-us.dict -hmm /usr/local/share/pocketsphinx/model/en-us/en-us -lm /usr/local/share/pocketsphinx/model/en-us/en-us.lm.bin ...

S1MB10T3 commented 6 years ago

Well back on working on my pi project. I can't seem figure out how to get the pocketsphinx built from source to work. I don't think it's installing in the right place. I'm going to try to see if I can get openassistant to work on a vm or a clean install of raspbian.