MycroftAI / mycroft-core

Mycroft Core, the Mycroft Artificial Intelligence platform.
https://mycroft.ai
Apache License 2.0
6.5k stars 1.27k forks source link

Feature: Allow users to prioritize skills for common play framework #2322

Open ChanceNCounter opened 4 years ago

ChanceNCounter commented 4 years ago

Down the road, Mycroft will likely support any number of media programs and services. It might be useful to let users specify the order in which they'd rather use a given skill or service when the common play framework is invoked.

For example, a user might have access to both Netflix and CBS All Access, both of which currently carry Star Trek: TNG. The common play framework prefers intents that consume more of the words in the user's request; both intents would consume all of the words in, "Play Star Trek The Next Generation."

If the user prefers one service over the other, it would be nice for Mycroft to lean in that direction, especially if the preferred intent doesn't report the higher-confidence match. This could be as "easy" (to design, not to implement) as draggable cards in the web interface.

CircuitByrd commented 4 years ago

This is an excellent enhancement request. We should also be careful to design to handle these types of events with the least user friction in mind, so when a priority hasn't been defined and all intents can be met by multiple applications, perhaps this could trigger a prompt that the user could respond to verbally: "Would you like to use App1 or App2 for this?" Optionally, after the user responds, Mycroft could mention that they can set a preference like you're describing for next time. This allows users to handle the conflict, and learn about the preferences setting, when and if they first encounter this scenario.

penrods commented 4 years ago

There is a "todo" in the framework for resolving instances where there are multiple Play providers that could handle the user's request. Currently the resolution is the simple/stupid "take the highest match", but I was ultimately thinking a better resolution would involve looking at the confidences and going into a resolution phase if there wasn't a clear winner. For example:

"play madonna" Spotify replies: yes, 90% Pandora replies: yes, 80%

In that case there isn't a clear winner. My original inclination was to use the best test of a user's intent -- ask the user! So start a dialog along the lines of:

I could play using either Pandora or Spotify. Which would you prefer? "Pandora" Now playing Madonna on Pandora...

As an added layer, we could track responses over time and start to offer defaults if the user picks that all the time.

Presenting these preferences in a graphical UI is also useful. The Play skill could show preference order like:

*Pandora
Spotify

Where the * indicates the preferred default, which maybe triggers when the confidence is >= the other service or some similar heuristic.

domcross commented 4 years ago

At least the Spotify and the (deprecated) Amazon-Music skill add a "bonus" to the confidence when the skill name is used in the request, e.g. "play Holiday by Madonna on Spotify" will most likely return a confidence of 100% for the Spotify skill.

ChanceNCounter commented 4 years ago

Indeed, but what if I want it to choose Spotify on its own, every single time, so long as Spotify replies with a high score? Next I decide I'd like it to prioritize Spotify --> YouTube Music, and I'm off to the proverbial races. Much easier, I think, to interfere with the parser based on some kind of user settings.

forslund commented 4 years ago

I can imagine a system similar to the registration of resting screens where the CPS skill sends a message (at startup or when requested) registering it's name along with it's skill id with the playback control skill.

Message('mycroft.play.register_name', data={'skill_id': self.skill_id, 'name': skill.name})

(or something similar)

The playback control skill can then catch these so it can know all existing CPS-skills and their "pretty" names ("spotify", "pandora", "youtube"} and can map them to a skill id returned by the CPS_match_query_phrase() methods.

Then an intent to list common play skills is added along with an intent for setting the prioritized skill...

The TODO mentioned by @penrods should also be implemented to allow users to disambiguate responses.