"Say" triggers speak skill always

ryanleesipes commented 8 years ago

"Hey Mycroft, What does Wikipedia say about Abraham Lincoln"

Mycroft responds: "About Abraham Lincoln"

MatthewScholefield commented 8 years ago

I would consider this an issue with adapt. Instead it should probably return a confidence of each match based on the length of the regex so that what does wikepedia say would always override say.

jgbreezer commented 7 years ago

I agree; last year I had a go at making a mycroft-like intent-parsing system (also in python) for a relatively small set of operations and added this weighting to match-confidences for the same reason. It worked well, though I also weighted matches on how close to the beginning of the phrase they were (or actually, based on where the next set of unparsed words started from the words received). Each skill would handle its phrase and return any leftover words at the end to be reparsed if needed, especially if it was part of a subclause separated with, for example. "and" or "then".

Note: I did try adding "sounds like" lists of words to each "skill" word-set that the TTS would sometimes return as alternates to improve match rates, then do a best-match and pro-rate the confidence appropriately, but it was getting too hard even with the limited set of features to cater for all recognised versions returned as a spare-time project.

clusterfudge commented 7 years ago

Adapt yields parse results in order of "most coverage of utterance", which is not necessarily in order of highest confidence. When computing confidence, regexes are weighted lower based on their inaccuracy, and all matches (regex or entity literal) have a confidence based on percentage of utterance represented. The latest release of Adapt (v0.3.0) will optionally expose the confidence of the individual tags within a parse result.

{u'ArticleTitle': u'say about foobar', 'intent_type': u'WikipediaIntent', 'confidence': 0.375, 'target': None, u'WikipediaKeyword': u'wikipedia'}

Stepping through this in a debugger, I can see that the 4th or 5th parse result yielded from adapt does in fact have a higher confidence than the speak intent, and triggers the wikipedia intent. The parse result is garbage (query: say about abraham lincoln), but Adapt isn't very good at parsing free text search queries out of natural language text. While regexes are a necessary evil here, they've been pretty heavily abused, and the more skills that use them, the higher likelihood of collisions like this.

At least for me, this bug was fixed by https://github.com/MycroftAI/mycroft-core/commit/4d6defd126c0a0b091718f8e15f8b033a84f9aa1

joshua-montgomery commented 7 years ago

This bug appears to be fixed.

MycroftAI / mycroft-core

"Say" triggers speak skill always #318