dscripka / openWakeWord

An open-source audio wake word (or phrase) detection framework with a focus on performance and simplicity.
Apache License 2.0
620 stars 59 forks source link

Error opening melspectrogram.tflite #148

Open asktimfisher opened 5 months ago

asktimfisher commented 5 months ago

Hi!

I've created a pretty sweet setup where I have openWakeWord running in the Linux environment right on a ChromeOS device. Works wonderfully. I've gone and kicked off tablet number 2 and went through my process but when executing my Python file (same as my working device) I get the following error:

ValueError: Could not open '/home/scdisplayai/.local/lib/python3.11/site-packages/openwakeword/resources/models/melspectrogram.tflite'.

I am not a Linux expert, so not a "Debian on ChromeOS" expert either. This feels somehow related to the how I've setup some packages (permissioning/etc.) but maybe not? Any thoughts?

Thanks!

asktimfisher commented 5 months ago

OK, so it turns out that I installed the latest version of openWakeWord on this new tablet (of course) which is v0.6.0 while I was running v0.5.1 on the first tablet. I uninstalled 6 and installed 5.1 on the newer tablet and all is resolved.

So it looks like the "resources" folder is missing entirely from the latest version.

Here's my short Python code that I'm running great with 5.1 but failing (as described in original post) in 6. Thoughts?

import pyaudio
import numpy as np
from openwakeword.model import Model
import requests
import os

# Variables
DEVICE = os.path.splitext(os.path.basename(__file__))[0] # Can also be specific like DEVICE = "tim_office"
SENSITIVITY = 0.6 # Need to find a way to set this via a pull from the NodeJS server
FORMAT = pyaudio.paInt16
CHANNELS = 1
RATE = 16000
DEVICE_INDEX = 0 # On Lenovo Chromebook: 0: VirtIO SoundCard: PCM 0 (hw:0,0), 1: sysdefault, 2: pipewire, 3: dmix (doesn't work), 4: default
CHUNK = 1280

# Setup microphone stream
audio = pyaudio.PyAudio()
mic_stream = audio.open(format=FORMAT, channels=CHANNELS, rate=RATE, input=True, frames_per_buffer=CHUNK)
# mic_stream = audio.open(format=FORMAT, channels=CHANNELS, rate=RATE, input=True, input_device_index=DEVICE_INDEX, frames_per_buffer=CHUNK)

# Load wakeword models
# All the options are found in model.py on Github: https://github.com/dscripka/openWakeWord/blob/main/openwakeword/model.py
owwModel = Model(wakeword_models=["/mnt/chromeos/GoogleDrive/SharedWithMe/330/chromemaster/hey_jarvis_v0.1.tflite", "/mnt/chromeos/GoogleDrive/SharedWithMe/330/chromemaster/lights_on_v0.1.tflite", "/mnt/chromeos/GoogleDrive/SharedWithMe/330/chromemaster/what_time_is_it_v0.1.tflite"])
# owwModel = Model()

# Capture loop
if __name__ == "__main__":
    print("Listening for wakewords...")

    last_detected = None  # Store the last detected wakeword

    while True:
        # Get audio data
        audio_data = np.frombuffer(mic_stream.read(CHUNK), dtype=np.int16)

        # Predict wakeword
        prediction = owwModel.predict(audio_data)

        for mdl, scores in owwModel.prediction_buffer.items():
            curr_score = scores[-1]  # Get the latest score

            # Check if score is above threshold and not recently detected
            if curr_score > SENSITIVITY and last_detected != mdl:
                formatted_score = f"{curr_score:.3f}"
                json_data = {
                    "wakeword": mdl,
                    "score": formatted_score,
                    "device": DEVICE
                }

                try:
                    response = requests.post("http://192.168.0.10:5555/api/start", json=json_data)
                    response.raise_for_status()
                except requests.RequestException as e:
                    print(f"Error sending data: {e}") # Add timestamp to beginning for clarity

                last_detected = mdl  # Update the last detected wakeword

        # Reset the last_detected if no wakeword is currently being detected
        if not any(score > 0.6 for scores in owwModel.prediction_buffer.values() for score in scores):
            last_detected = None