Help wanted: extending functionality

[drftg]

Hi,

I have been trying to extend some functionality. Not being a native English speaker I really wanted some short cuts to basic functions using just a snowboy hotword. Most wanted was turning the screen on after just uttering "mirror" in my own language. Kind of like a PIR but voice activated. To get this to work I changed a few lines.

In MMM-Assistant.js I extended the function:

  hotwordDetected : function (type) {
    // Start Google Assistant
    if (type.hotword == 'ASSISTANT') {
      this.sendSocketNotification('ACTIVATE_ASSISTANT')
      this.status = 'ACTIVATE_ASSISTANT'
    // Start command mode
    } else if (type.hotword == 'COMMAND') {
      this.sendSocketNotification('ACTIVATE_COMMAND')
      this.status = 'ACTIVATE_COMMAND'
    // Start snowboy hotword activated shell command
    } else if (type.hotword == 'EXECUTE') {
      this.sendSocketNotification('EXECUTE', this.config.snowboy.models[type.index-1].parameter)
     // kludge, don't know how else to get back to listening
      this.sendSocketNotification('SPEAK', {text: ''}) 
    // Send snowboy hotword activated notification to all modules
    }  else if (type.hotword == 'NOTIFY') {
     // kludge, don't know how else to get back to listening
      this.sendSocketNotification('SPEAK', {text: this.config.snowboy.models[type.index-1].parameter}) 
      this.sendNotification(this.config.snowboy.models[type.index-1].parameter)
    }
  },

And in node_helper.js, I added to the function socketNotificationReceived: function the following:

      case 'EXECUTE':
        execute(payload, function(callback) {
          console.log(callback)
        })
        break

and in activateHotword: function(), I changed this line:

      this.sendSocketNotification('HOTWORD_DETECTED', {hotword:hotword, index:index})

So now you can extend the snowboy models in your config to something like this:

        snowboy: {
            models: [
                {
                    file: "resources/show_mirror.pmdl",
                    sensitivity: 0.45,
                    parameter: 'vcgencmd display_power 1', // monitor on
                    hotwords : "EXECUTE"         
                },
                {
                    file: "resources/hide_mirror.pmdl",
                    sensitivity: 0.5,
                    parameter: 'vcgencmd display_power 0',
                    hotwords : "EXECUTE"              
                },
                {
                    file: "resources/alert.pmdl",
                    sensitivity: 0.5,
                    parameter: 'SHOW_ALERT',
                    hotwords : "NOTIFY"  
                },

Now my problem is the kludge I used in MMM-Assistant.js. I don't know how I can gracefully revert to the default state that just listens for another hotword. Any help is appreciated.

[drftg]

Solved

[E3V3A]

@drftg This sound pretty cool, but I'd like to understand better how it works.

• For example, if you turn on the screen with show_mirror.pmdl, in what state are you in? Does it expect you to say something more, or it stays as before, and still listens for the hotword?

• Also, I noticed you changed the sensitivity for some reason. Why?

[drftg]

@E3V3A The trouble I had was exactly that, the state. What you want is just a reaction to the hotword and then return to the listening state without further interaction. That I solved. I even added the possibility to use a custom confirmation sound instead of dong.wav. So now I can use friendly "Yes?" or a yawn ;-)

I am tweaking the sensitivity a bit because I am using pmdl's of short words with just a few training sessions. They are sometimes too easily recognized (false positive). Even alexa.umdl sometimes fires when I just cough.

At this moment I am very happy with the changes I made. Just uttering "mirror" turns the screen on. What I have yet to find out is how to set a timer for turning the screen off again. I was hoping to find another module that I can send a notification to to handle the screen. That is why I added the "NOTIFY" option. I was looking at MMM-PIR and others. Sending "USER_PRESENCE" should do it but most solutions rely on an actual PIR being present.