Distinguishing between Intents when the same entity type occurs in both

[Stikemanley]

Distinguishing between Luis Intents when the same entity type occurs in both

Issue

I'm having trouble with my LUIS model distinguishing between two intents, a generic search intent which leads off to a query of external systems and a person lookup which leads to a database query.

---

How I think it should work

I have two intents, Generic Search and People Lookup I have three entities People, ContentType, and Systems I would like Luis to direct the query Stevie Wonder to People Lookup and the query Search all issues regarding some topic assigned to Stevie Wonder to Generic Search

What is happening

Whenever I attempt to do a person lookup consisting of just the individual's name, unless that person is included in my People Lookup utterances literally it is categorized as a Generic Search.

What we'd like to know

Will someone please help me identify the best practice for separating intents such as these where the utterances will be short and there is a shared entity? Shouldn't an utterance of an example entity be sufficient to cover all queries consisting of just that entity? It doesn't seem logical that we would have to include everyone's name as an utterance for People Lookup as well as an entity.

Bot Framework Included

I've included a bot framework where I was able to recreate the issue. If I search show me issues assigned to emily it identifies it as a Generic Search Intent. However, if I just search betty It is also labeled as a Generic Search Intent.

Example_Bot.txt

[Stikemanley]

Is there an issue with this post? My question remains, I would like this post open so it has an opportunity of being seen by the community.

Thank you,

On Wed, Feb 28, 2018 at 3:43 PM, Chris Mullins notifications@github.com wrote:

Closed #76 https://github.com/Microsoft/Cognitive-LUIS-Windows/issues/76 .

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Microsoft/Cognitive-LUIS-Windows/issues/76#event-1497632883, or mute the thread https://github.com/notifications/unsubscribe-auth/AWQn0jUgmt1fuuqQh-ZtCTl23mc7Kh7Mks5tZbqPgaJpZM4SHnkG .

[cleemullins]

We're cleaning up the entire code-base in this repo. We effectively just scrapped the old code, and are replace it with the code from the BotBuilder v3 C# repo (and then the JS code).

Due to how stale this repo became, I just bulk-closed everything figuring a fresh start is the best choice.

As as aside, the LUIS team is working towards a solution to this. This morning I reviewed the spec around this, and Emad is working on implementing the spec. That code will be showing up in this repo at some time in the near future.

[stevengum]

@Stikemanley what this stems from is that the intent classification doesn't see that "Stevie Wonder" is a People-type entity. Entity extraction and intent classification aren't directly coupled by the models; therefore if you add 'Franklin Delano Roosevelt', 'Barrack Obama', 'Bill Gates', 'Satya Nadella' as People-type entities and add these specific utterances (just the names) as the People Lookup intent, someone typing 'Stevie Wonder' (which is also a People type entity) won't necessarily lead to the correct intent of People Lookup.

One way to potentially improve the People Lookup scores for names would be to add a few more name combinations for this intent, but to also add more training data with longer utterances for other intents, like Generic Search. This should improve the recognition for low-number-of-token-utterances, which in this case are utterances consisting of just the name of a person.

Another path to potentially pursue is to setup a filter on the LUIS results in your bot. This way if you get the wrong intent, but the utterance consists of only a extracted entity, you can instead switch the top scoring intent to People Lookup, or the corresponding intent. I'm not sure if there are builtin methods that allow you to do this, but what a partial implementation of a filter would look like in Node can be found here. You could then use the Programmatic APIs to add these names/utterances as training data for the People Lookup intent. A code snippet in C# can be found at the bottom of the page.

[Stikemanley]

Amazing,

Thanks for the response guys, I'll look into implementing some of these changes in our model.

Best,

On Wed, Feb 28, 2018 at 5:44 PM, Steven Gum notifications@github.com wrote:

@Stikemanley https://github.com/stikemanley what this stems from is that the intent classification doesn't see that "Stevie Wonder" is a People-type entity. Entity extraction and intent classification aren't directly coupled by the models; therefore if you add 'Franklin Delano Roosevelt', 'Barrack Obama', 'Bill Gates', 'Satya Nadella' as People-type entities and add these specific utterances (just the names) as the People Lookup intent, someone typing 'Stevie Wonder' (which is also a People type entity) won't necessarily lead to the correct intent of People Lookup.

One way to potentially improve the People Lookup scores for names would be to add a few more name combinations for this intent, but to also add more training data with longer utterances for other intents, like Generic Search. This should improve the recognition for low-number-of-token-utterances, which in this case are utterances consisting of just the name of a person.

Another path to potentially pursue is to setup a filter on the LUIS results in your bot. This way if you get the wrong intent, but the utterance consists of only a extracted entity, you can instead switch the top scoring intent to People Lookup, or the corresponding intent. I'm not sure if there are builtin methods that allow you to do this, but what a partial implementation of a filter would look like in Node can be found here https://github.com/Microsoft/BotBuilder/blob/master/Node/examples/feature-onFilter/app.js#L67. You could then use the Programmatic APIs https://westus.dev.cognitive.microsoft.com/docs/services/5890b47c39e2bb17b84a55ff/operations/5890b47c39e2bb052c5b9c08 to add these names/utterances as training data for the People Lookup intent. A code snippet in C# can be found at the bottom of the page.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Microsoft/Cognitive-LUIS-Windows/issues/76#issuecomment-369409895, or mute the thread https://github.com/notifications/unsubscribe-auth/AWQn0pSMlvSDQTvoG2w2V2C7u7wxptRaks5tZdbHgaJpZM4SHnkG .