JamesBrill / react-speech-recognition

💬Speech recognition for your React app
https://webspeechrecognition.com/
MIT License
657 stars 119 forks source link

Multiple commands using the same callback #70

Closed Nitzahon closed 3 years ago

Nitzahon commented 3 years ago

What is the best way to do this, do I have to define multiple commands with the same callback, or can a command be an array of possible strings which will call the same callback?

Nitzahon commented 3 years ago

so it seems you can put an array of commands to use the same callback

JamesBrill commented 3 years ago

Array commands are not actually supported (though they probably should be). See this comment on your other issue.

JamesBrill commented 3 years ago

Reopening to remind myself to add support for array commands.

JamesBrill commented 3 years ago

@Nitzahon Array-based commands are now supported in v3.4.0

Nitzahon commented 3 years ago

You sir, are what is known as a mensch

Nitzahon commented 3 years ago

One question though, how do you send the spoken command itself from the array as the callbacks argument? like this:

    {
      command: ['Hello', 'Hi'],
      callback: () => setMessage(command),
      matchInterim: true
    }

or like this:

    {
      command: ['Hello', 'Hi'],
      callback: () => setMessage(command, spokenPhrase),
      matchInterim: true
    }

or something else? will I still need to use the same solution you gave me in 3.3.0?


  const videoCommandCallback = (command, spokenPhrase) => {
    sendAns(command)
    handleReset()

  }
  const videoCommands = ['Hello', 'Hi'].map(phrase => ({
    command: phrase,
    callback: videoCommandCallback,
    isFuzzyMatch: true,
    fuzzyMatchingThreshold: 0.6
  }))
Nitzahon commented 3 years ago

so this works:

  const videoCommandCallback = (command, spokenPhrase) => {
    sendAns(command)
    handleReset()

  }
const commands =[
    {
      command: ['Hello', 'Hi'],
      callback: videoCommandCallback,
      isFuzzyMatch: true,
      fuzzyMatchingThreshold: 0.6,
    }
]

There is only the issue, of, if multiple commands match the fuzzy requirements, all are called with the callback instead of the command which most closely matches. i.e.: If my array contains the commands ["Everything is working", "Nothing is working"], my fuzzyMatchingThreshold is set to 0.7 or lower and I say "Everything is working" The callback will be called twice, once for "Everything is working", and once for "Nothing is working", even though "Everything is working" more closely matches.

JamesBrill commented 3 years ago

Ah yes, it would make sense to know which of the commands in the array was matched. I'll make a quick update.

JamesBrill commented 3 years ago

@Nitzahon Give v3.5.0 a try. The callback can get the matched command from the last argument, which will be an object with a new property: command.

JamesBrill commented 3 years ago

There is only the issue, of, if multiple commands match the fuzzy requirements, all are called with the callback instead of the command which most closely matches. i.e.: If my array contains the commands ["Everything is working", "Nothing is working"], my fuzzyMatchingThreshold is set to 0.7 or lower and I say "Everything is working" The callback will be called twice, once for "Everything is working", and once for "Nothing is working", even though "Everything is working" more closely matches.

This is an interesting point. It may be desirable in some cases to just trigger the callback on the best match when fuzzy matching is used. If this would be a useful feature for you, let me know and I can look into it.

Nitzahon commented 3 years ago

There is only the issue, of, if multiple commands match the fuzzy requirements, all are called with the callback instead of the command which most closely matches. i.e.: If my array contains the commands ["Everything is working", "Nothing is working"], my fuzzyMatchingThreshold is set to 0.7 or lower and I say "Everything is working" The callback will be called twice, once for "Everything is working", and once for "Nothing is working", even though "Everything is working" more closely matches.

This is an interesting point. It may be desirable in some cases to just trigger the callback on the best match when fuzzy matching is used. If this would be a useful feature for you, let me know and I can look into it.

Letting you know, this would be useful ;-)

Nitzahon commented 3 years ago

@Nitzahon Give v3.5.0 a try. The callback can get the matched command from the last argument, which will be an object with a new property: command.

tried this. Did not work:

    {
      command: videoCommandPhrases.commands,
      callback: ({ command }) => console.log("command is: "+command),
      isFuzzyMatch: true,
      fuzzyMatchingThreshold: 0.6,
    }

expected outout: "command is: Everything is Working" actual output: "command is: undefined"

JamesBrill commented 3 years ago

I'll repro this and see what I can do.

JamesBrill commented 3 years ago

Initial tests indicate this issue is exclusive to fuzzy-matched commands. My bad for not testing that case when I made that release. Investigating.

JamesBrill commented 3 years ago

Ah wait, this does make sense. Fuzzy callbacks get three other arguments before that one.

JamesBrill commented 3 years ago

With fuzzy commands, you can use the first arg like before:

    {
      command: ["Everything is working", "Nothing is working"],
      callback: (command) => console.log("command is: "+command),
      isFuzzyMatch: true,
      fuzzyMatchingThreshold: 0.6,
    },
JamesBrill commented 3 years ago

This does make me want to clean up the API a bit. Ideally, there would just be one object arg for both fuzzy and non-fuzzy commands, and the callback could unpack whatever it needed. It feels messy having command in two places for fuzzy commands.

Before I do that, I'm going to look into your request for "closest match wins" for fuzzy commands.

JamesBrill commented 3 years ago

@Nitzahon Give v3.6.0 a try - it adds an option to only trigger the callback once, on the best fuzzy match.