Call start() again after terminate()

[kgorszczyk]

Hi,

im trying to add snowboy to a personal assistant app i've been working on but to be able to record the audio i first need to terminate all recording streams from snowboy otherwise i get an Audio Deivce Busy Error.

After accessing detector.terminate() I can finally record my command and the python file processes it correctly. My only problem is, that while the prog loops detector.start(...) does not work anymore.

Is there a way to reenable detector.start(...) so snowboy listens for the Hotword and then kill everything with detector.terminate() (And so on and on)?

Thanks :)

[xuchen]

Yes that should be easy, just move the last two lines in HotwordDetector.__init__() of snowboydecoder.py:

    self.audio = pyaudio.PyAudio()
    self.stream_in = self.audio.open(
        input=True, output=False,
        format=self.audio.get_format_from_width(
            self.detector.BitsPerSample() / 8),
        channels=self.detector.NumChannels(),
        rate=self.detector.SampleRate(),
        frames_per_buffer=2048,
        stream_callback=audio_callback)

down to the start of the start() function, then you can repeatedly call terminate() and start()!

[kgorszczyk]

Thanks it worked perfectly! I'm currently running it on a Raspberry Pi and for quite some time i get these errors after running the HotWord detection:

ALSA lib pcm_dmix.c:1022:(snd_pcm_dmix_open) unable to open slave ALSA lib pcm_dmix.c:1022:(snd_pcm_dmix_open) unable to open slave ALSA lib pcm_dmix.c:1022:(snd_pcm_dmix_open) unable to open slave ALSA lib pcm_dmix.c:1022:(snd_pcm_dmix_open) unable to open slave ALSA lib pcm_dmix.c:1022:(snd_pcm_dmix_open) unable to open slave ALSA lib pcm_dmix.c:1022:(snd_pcm_dmix_open) unable to open slave ALSA lib pcm_dmix.c:1022:(snd_pcm_dmix_open) unable to open slave ALSA lib pcm_dmix.c:1022:(snd_pcm_dmix_open) unable to open slave Cannot connect to server socket err = No such file or directory Cannot connect to server request channel jack server is not running or cannot be started

Do you have an Idea what that could be?

[xuchen]

Does it work at all? Often we see similar messages but the code still works. Then in this case we ignore it. If you have specific error that stops the code from running, could you please paste it here?

[kgorszczyk]

No the code still works even with those warning messages. 👍 Thanks.

[peterGruen]

What exact position do you mean with:" down to the start of the start() function, then you can repeatedly call terminate() and start()! I get some error: global name 'audio_callback' is not defined thank you

[chenguoguo]

You can move the callback function (starting from line 77) to the start() function as well.

[peterGruen]

Thank you, now it works !

[tbass134]

forgive me for bringing this up again, but I am unable to get this working. If i move the frames_per_buffer and stream_callback to the start() function, no hotwords are detected. Here's the init of HotwordDetector

def __init__(self, decoder_model,
                 resource=RESOURCE_FILE,
                 sensitivity=[],
                 audio_gain=1):

and here is start method

 def start(self, detected_callback=play_audio_file,
              interrupt_check=lambda: False,
              sleep_time=0.03,
              frames_per_buffer=2048,
              stream_callback=audio_callback):

the audio_callback method is

    def audio_callback(in_data, frame_count, time_info, status):
            self.ring_buffer.extend(in_data)
            play_data = chr(0) * len(in_data)
            return play_data, pyaudio.paContinue

[chenguoguo]

@tbass134 you misunderstood it...

What you should do:

1. First delete the following code block from the __init__ function

       self.audio = pyaudio.PyAudio()
       self.stream_in = self.audio.open(
           input=True, output=False,
           format=self.audio.get_format_from_width(
               self.detector.BitsPerSample() / 8),
           channels=self.detector.NumChannels(),
           rate=self.detector.SampleRate(),
           frames_per_buffer=2048,
           stream_callback=audio_callback)

2. Then add the same code block to the start function

   def start(self, detected_callback=play_audio_file,
             interrupt_check=lambda: False,
             sleep_time=0.03):
       """
       Start the voice detector. For every `sleep_time` second it checks the
       audio buffer for triggering keywords. If detected, then call
       corresponding function in `detected_callback`, which can be a single
       function (single model) or a list of callback functions (multiple
       models). Every loop it also calls `interrupt_check` -- if it returns
       True, then breaks from the loop and return.
   
       :param detected_callback: a function or list of functions. The number of
                                 items must match the number of models in
                                 `decoder_model`.
       :param interrupt_check: a function that returns True if the main loop
                               needs to stop.
       :param float sleep_time: how much time in second every loop waits.
       :return: None
       """
       self.audio = pyaudio.PyAudio()
       self.stream_in = self.audio.open(
           input=True, output=False,
           format=self.audio.get_format_from_width(
               self.detector.BitsPerSample() / 8),
           channels=self.detector.NumChannels(),
           rate=self.detector.SampleRate(),
           frames_per_buffer=2048,
           stream_callback=audio_callback)
   
       if interrupt_check():
           logger.debug("detect voice return")
           return
   
       tc = type(detected_callback)
   
       ........

Then you can call terminate and start alternatively.

[tbass134]

ah thank you @chenguoguo ! sorry i misunderstood your initial fix.

[hdi200]

@chenguoguo I'm sorry about this but I think I'm missing something. I did exactly what you said in your answer to @tbass134 but I'm still getting an error: "global name 'audio_callback' is not defined". Here is the beginning of my start function does this look right? Thank you so much in advanced!

    :param detected_callback: a function or list of functions. The number of
                              items must match the number of models in
                              `decoder_model`.
    :param interrupt_check: a function that returns True if the main loop
                            needs to stop.
    :param float sleep_time: how much time in second every loop waits.
    :return: None
    """

    self.audio = pyaudio.PyAudio()
    self.stream_in = self.audio.open(
        input=True, output=False,
        format=self.audio.get_format_from_width(
            self.detector.BitsPerSample() / 8),
        channels=self.detector.NumChannels(),
        rate=self.detector.SampleRate(),
        frames_per_buffer=2048,
        stream_callback=audio_callback)

`

[chenguoguo]

You can move the audio_callback function to the start function as well.

[sujitss]

@chenguoguo Sorry for asking but even after moving the aforesaid two lines and audio_callback function in start() function, I am getting an error as follow

Traceback (most recent call last): File "okay.py", line 11, in <module> detector.start(detected_callback) File "/home/pi/rpi-arm-raspbian-8.0-1.1.0/snowboydecoder.py", line 144, in start tm = type(decoder_model) UnboundLocalError: local variable 'decoder_model' referenced before assignment

[chenguoguo]

@sujitss please post your code so that we can take a look.

[sujitss]

@chenguoguo Here is my snowboydecoder.py

`#!/usr/bin/env python

import collections import pyaudio import snowboydetect import time import wave import os import logging

logging.basicConfig() logger = logging.getLogger("snowboy") logger.setLevel(logging.INFO) TOP_DIR = os.path.dirname(os.path.abspath(file))

RESOURCE_FILE = os.path.join(TOP_DIR, "resources/common.res") DETECT_DING = os.path.join(TOP_DIR, "resources/ding.wav") DETECT_DONG = os.path.join(TOP_DIR, "resources/dong.wav")

class RingBuffer(object): """Ring buffer to hold audio from PortAudio""" def init(self, size = 4096): self._buf = collections.deque(maxlen=size)

def extend(self, data):
    """Adds data to the end of buffer"""
    self._buf.extend(data)

def get(self):
    """Retrieves data from the beginning of buffer and clears it"""
    tmp = bytes(bytearray(self._buf))
    self._buf.clear()
    return tmp

def play_audio_file(fname=DETECT_DING): """Simple callback function to play a wave file. By default it plays a Ding sound.

:param str fname: wave file name
:return: None
"""
ding_wav = wave.open(fname, 'rb')
ding_data = ding_wav.readframes(ding_wav.getnframes())
audio = pyaudio.PyAudio()
stream_out = audio.open(
    format=audio.get_format_from_width(ding_wav.getsampwidth()),
    channels=ding_wav.getnchannels(),
    rate=ding_wav.getframerate(), input=False, output=True)
stream_out.start_stream()
stream_out.write(ding_data)
time.sleep(0.2)
stream_out.stop_stream()
stream_out.close()
audio.terminate()

class HotwordDetector(object): """ Snowboy decoder to detect whether a keyword specified by decoder_model exists in a microphone input stream.

:param decoder_model: decoder model file path, a string or a list of strings
:param resource: resource file path.
:param sensitivity: decoder sensitivity, a float of a list of floats.
                          The bigger the value, the more senstive the
                          decoder. If an empty list is provided, then the
                          default sensitivity in the model will be used.
:param audio_gain: multiply input volume by this factor.
"""
def __init__(self, decoder_model,
             resource=RESOURCE_FILE,
             sensitivity=[],
             audio_gain=1):

    """def audio_callback(in_data, frame_count, time_info, status):
        self.ring_buffer.extend(in_data)
        play_data = chr(0) * len(in_data)
        return play_data, pyaudio.paContinue

    tm = type(decoder_model)
    ts = type(sensitivity)
    if tm is not list:
        decoder_model = [decoder_model]
    if ts is not list:
        sensitivity = [sensitivity]
    model_str = ",".join(decoder_model)

    self.detector = snowboydetect.SnowboyDetect(
        resource_filename=resource.encode(), model_str=model_str.encode())
    self.detector.SetAudioGain(audio_gain)
    self.num_hotwords = self.detector.NumHotwords()

    if len(decoder_model) > 1 and len(sensitivity) == 1:
        sensitivity = sensitivity*self.num_hotwords
    if len(sensitivity) != 0:
        assert self.num_hotwords == len(sensitivity), \
            "number of hotwords in decoder_model (%d) and sensitivity " \
            "(%d) does not match" % (self.num_hotwords, len(sensitivity))
    sensitivity_str = ",".join([str(t) for t in sensitivity])
    if len(sensitivity) != 0:
        self.detector.SetSensitivity(sensitivity_str.encode())

    self.ring_buffer = RingBuffer(
        self.detector.NumChannels() * self.detector.SampleRate() * 5)"""

def start(self, detected_callback=play_audio_file,
          interrupt_check=lambda: False,
          sleep_time=0.03):
    """
    Start the voice detector. For every `sleep_time` second it checks the
    audio buffer for triggering keywords. If detected, then call
    corresponding function in `detected_callback`, which can be a single
    function (single model) or a list of callback functions (multiple
    models). Every loop it also calls `interrupt_check` -- if it returns
    True, then breaks from the loop and return.

    :param detected_callback: a function or list of functions. The number of
                              items must match the number of models in
                              `decoder_model`.
    :param interrupt_check: a function that returns True if the main loop
                            needs to stop.
    :param float sleep_time: how much time in second every loop waits.
    :return: None
    """
    def audio_callback(in_data, frame_count, time_info, status):
        self.ring_buffer.extend(in_data)
        play_data = chr(0) * len(in_data)
        return play_data, pyaudio.paContinue

    tm = type(decoder_model)
    ts = type(sensitivity)
    if tm is not list:
        decoder_model = [decoder_model]
    if ts is not list:
        sensitivity = [sensitivity]
    model_str = ",".join(decoder_model)

    self.detector = snowboydetect.SnowboyDetect(
        resource_filename=resource.encode(), model_str=model_str.encode())
    self.detector.SetAudioGain(audio_gain)
    self.num_hotwords = self.detector.NumHotwords()

    if len(decoder_model) > 1 and len(sensitivity) == 1:
        sensitivity = sensitivity*self.num_hotwords
    if len(sensitivity) != 0:
        assert self.num_hotwords == len(sensitivity), \
            "number of hotwords in decoder_model (%d) and sensitivity " \
            "(%d) does not match" % (self.num_hotwords, len(sensitivity))
    sensitivity_str = ",".join([str(t) for t in sensitivity])
    if len(sensitivity) != 0:
        self.detector.SetSensitivity(sensitivity_str.encode())

    self.ring_buffer = RingBuffer(
        self.detector.NumChannels() * self.detector.SampleRate() * 5)

    self.audio = pyaudio.PyAudio()
    self.stream_in = self.audio.open(
        input=True, output=False,
        format=self.audio.get_format_from_width(
            self.detector.BitsPerSample() / 8),
        channels=self.detector.NumChannels(),
        rate=self.detector.SampleRate(),
        frames_per_buffer=2048,
        stream_callback=audio_callback)

    if interrupt_check():
        logger.debug("detect voice return")
        return

    tc = type(detected_callback)
    if tc is not list:
        detected_callback = [detected_callback]
    if len(detected_callback) == 1 and self.num_hotwords > 1:
        detected_callback *= self.num_hotwords

    assert self.num_hotwords == len(detected_callback), \
        "Error: hotwords in your models (%d) do not match the number of " \
        "callbacks (%d)" % (self.num_hotwords, len(detected_callback))

    logger.debug("detecting...")

    while True:
        if interrupt_check():
            logger.debug("detect voice break")
            break
        data = self.ring_buffer.get()
        if len(data) == 0:
            time.sleep(sleep_time)
            continue

        ans = self.detector.RunDetection(data)
        if ans == -1:
            logger.warning("Error initializing streams or reading audio data")
        elif ans > 0:
            message = "Keyword " + str(ans) + " detected at time: "
            message += time.strftime("%Y-%m-%d %H:%M:%S",
                                     time.localtime(time.time()))
            logger.info(message)
            callback = detected_callback[ans-1]
            if callback is not None:
                callback()

    logger.debug("finished.")

def terminate(self):
    """
    Terminate audio stream. Users cannot call start() again to detect.
    :return: None
    """
    self.stream_in.stop_stream()
    self.stream_in.close()
    self.audio.terminate()`

and here is my main file

` import snowboydecoder import subprocess import os.path def detected_callback(): detector.terminate() print "yes, how may I help you" os.system('arecord -D plughw:0,0 -f cd -c 1 -t wav -d 3 -q -r 16000 | flac - -s -f --best --sample-rate 16000 -o test.flac) subprocess.call(['./speech-rec1.sh'])

detector = snowboydecoder.HotwordDetector("okay.pmdl", sensitivity=0.5, audio_gain=1) detector.start(detected_callback) `

[chenguoguo]

@sujitss you copied too much into your start() function, you can delete the following block from it I believe:

    tm = type(decoder_model)
    ts = type(sensitivity)
    if tm is not list:
        decoder_model = [decoder_model]
    if ts is not list:
        sensitivity = [sensitivity]
    model_str = ",".join(decoder_model)

    self.detector = snowboydetect.SnowboyDetect(
        resource_filename=resource.encode(), model_str=model_str.encode())
    self.detector.SetAudioGain(audio_gain)
    self.num_hotwords = self.detector.NumHotwords()

    if len(decoder_model) > 1 and len(sensitivity) == 1:
        sensitivity = sensitivity*self.num_hotwords
    if len(sensitivity) != 0:
        assert self.num_hotwords == len(sensitivity), \
            "number of hotwords in decoder_model (%d) and sensitivity " \
            "(%d) does not match" % (self.num_hotwords, len(sensitivity))
    sensitivity_str = ",".join([str(t) for t in sensitivity])
    if len(sensitivity) != 0:
        self.detector.SetSensitivity(sensitivity_str.encode())

    self.ring_buffer = RingBuffer(
        self.detector.NumChannels() * self.detector.SampleRate() * 5)

[sujitss]

@chenguoguo Thanks, it's finally working and yes kudos for this amazing stuff.

[b-lenin]

I am using snowboy and speech recognition . I have done all the things u said but still i am getting the error as " IoError : [Errno -9985] Device unavailable

[AliBigdeli]

well i actually have the same probelm when i recall the funtion to hear the sound to detect hotword it ends up with same error IoError : [Errno -9985] Device unavailable those solutions which are up there are not useful anymore they were useful for the previous version not this one can anyone make this clear and straight forward what should be done to make this work?

[irthomasthomas]

@AliBigdeli Did you solve this in the end? You need to copy the new line with no_alsa_error():

[AliBigdeli]

@AliBigdeli Did you solve this in the end? You need to copy the new line with no_alsa_error():

yes i did, by passing the object to all the functions kinda dirty way but it worked and btw i am looking for another alternative way i have seen a youtube video which talks about tensorflow and training a data set for voice i guess ill be going for that cause i need universal samples more than private ones but still its a challenge cause i havent seen a good module as good as this one.

[kabcasa]

chenguoguo i get this message when i try your snowboydecoder : Traceback (most recent call last): File "model-test.py", line 10, in detector = snowboydecoder2.HotwordDetector("green.pmdl", sensitivity=0.5, au dio_gain=1) File "/home/pi/rpi-arm-raspbian-8.0-1.1.1/rpi-arm-raspbian-8.0-1.1.1/snowboyde coder2.py", line 112, in init self.detector.ApplyFrontend(apply_frontend) File "/home/pi/rpi-arm-raspbian-8.0-1.1.1/rpi-arm-raspbian-8.0-1.1.1/snowboyde tect.py", line 97, in getattr = lambda self, name: _swig_getattr(self, SnowboyDetect, name) File "/home/pi/rpi-arm-raspbian-8.0-1.1.1/rpi-arm-raspbian-8.0-1.1.1/snowboyde tect.py", line 74, in _swig_getattr return _swig_getattr_nondynamic(self, class_type, name, 0) File "/home/pi/rpi-arm-raspbian-8.0-1.1.1/rpi-arm-raspbian-8.0-1.1.1/snowboyde tect.py", line 69, in _swig_getattr_nondynamic return object.getattr(self, name) AttributeError: type object 'object' has no attribute 'getattr'