search

wit-ai / wit

Natural Language Interface for apps and devices
https://wit.ai/
931 stars 91 forks source link

contact entity needs some filtering #894

Closed bmcclenahan closed 6 years ago

bmcclenahan commented 6 years ago

Do you want to request a feature, report a bug, or ask a question about wit? bug

What is the current behavior? the built-in contact entity frequently returns grammatical elements like prepositions and articles (on, the) from the text

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem. Who's on the roster on December 31 returns who on and the as contact entity values

What is the expected behavior? the contact entity should filter out values that are obvious grammatical elements

What is the App ID where you are experiencing this issue (if applicable)? 58aecb8b-e2a7-46ff-8f5d-3a04e81575ac

patapizza commented 6 years ago

Thanks for reporting @bmcclenahan, I'll improve the model soon.

patapizza commented 6 years ago

@bmcclenahan This is now fixed.

bmcclenahan commented 6 years ago

the contact entity is still a bit indiscriminate and greedy about what it claims in a sentence. Who's on the service team tonight? returns Who's as a contact Who is on the service team tonight? returns service team as a contact

contact is very important to my application, and it's just too variable. most times it's very accurate and trustworthy and then all of a sudden it it's just random and rubbish.

I'm having to cover these in code exceptions, and if I need to write many more, it's not going to be worth using wit - I may as well just do it all in code.

blandinw commented 6 years ago

cc @patapizza to have a look at feasibility @bmcclenahan it would be helpful to have more erroneous predictions to find patterns

patapizza commented 6 years ago

@bmcclenahan I feel you. I'll look into improving wit/contact this week. It would help a lot if you can provide a bunch of false positives. Thanks!

bmcclenahan commented 6 years ago

I'll see what I can find I have a feeling that I've retrained some of them and thus the trace of the error has gone Bruce

From: Julien Odent notifications@github.com Sent: Tuesday, 9 January 2018 4:15 AM To: wit-ai/wit Cc: bmcclenahan; Mention Subject: Re: [wit-ai/wit] contact entity needs some filtering (#894)

@bmcclenahanhttps://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fbmcclenahan&data=02%7C01%7C%7Ca70ff5fc39514de30ad608d556c3c4cc%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636510321179179924&sdata=ExbpLUmsEThj%2BIP83NWeKeUNZaftT%2FbbRunIzLRYnQk%3D&reserved=0 I feel you. I'll look into improving wit/contact this week. It would help a lot if you can provide a bunch of false positives. Thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fwit-ai%2Fwit%2Fissues%2F894%23issuecomment-356048235&data=02%7C01%7C%7Ca70ff5fc39514de30ad608d556c3c4cc%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636510321179179924&sdata=%2FEM%2BSOi97MkPOjm8OnfZ6UG9jdsvotMyoGjzr6aCbAc%3D&reserved=0, or mute the threadhttps://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAYz9TfmskS19rn6gwJNM8RuvPFouU3sbks5tIlsygaJpZM4RJ0vR&data=02%7C01%7C%7Ca70ff5fc39514de30ad608d556c3c4cc%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636510321179179924&sdata=obsAP%2FYO%2Fcycgw9xGd3XrA5AGV4BLFZKHJ9GLDUTBoE%3D&reserved=0.

bmcclenahan commented 6 years ago

Some samples of contact being returned incorrectly

[ { "entities": { "planitem": [ { "confidence": 1 "start": 16 "end": 20 "body": " plan " "value": { "value": " plan " } "entity": " planitem " } ] "contact": [ { "confidence": 0.9315 "start": 8 "end": 15 "body": " service " "value": { "value": " service " } "entity": " contact " } ] "intent": [ { "confidence": 0.99992361664161 "value": { "value": " service_query " } "entity": " intent " } ] } "confidence": null "_text": " See the service plan " "intent": " default_intent " "intent_id": " default_intent_id " } ]

{ "entities": { "datetime": [ { "confidence": 0.961075 "start": 20 "end": 27 "body": " tonight " "value": { } "entity": " datetime " } ] "planitem": [ { "confidence": 0.99594153298319 "start": 11 "end": 15 "body": " plan " "value": { "value": " plan " } "entity": " planitem " } ] "contact": [ { "confidence": 0.93691 "start": 0 "end": 6 "body": " What’s " "value": { "value": " What’s " } "entity": " contact " } ] "intent": [ { "confidence": 0.99981033913868 "value": { "value": " service_query " } "entity": " intent " } ] } "confidence": null "_text": " What’s the plan for tonight? " "intent": " default_intent " "intent_id": " default_intent_id " } ]

[ { "entities": { "datetime": [ { "confidence": 0.9648375 "start": 15 "end": 25 "body": " on tonight " "value": { } "entity": " datetime " } ] "contact": [ { "confidence": 0.85268 "start": 0 "end": 5 "body": " Who’s " "value": { "value": " Who’s " } "entity": " contact " } ] "position": [ { "confidence": 0.75470433635482 "start": 6 "end": 14 "body": " rostered " "value": { "value": " roster " } "entity": " position " } ] "intent": [ { "confidence": 0.99999990521053 "value": { "value": " roster_query " } "entity": " intent " } ] } "confidence": null "_text": " Who’s rostered on tonight? " "intent": " default_intent " "intent_id": " default_intent_id " } ]

[ { "entities": { "datetime": [ { "confidence": 0.9651 "start": 15 "end": 23 "body": " tomorrow " "value": { "value": " 2017-12-31T00:00:00.000-06:00 " } "entity": " datetime " } ] "contact": [ { "confidence": 0.90841 "start": 0 "end": 5 "body": " Whose " "value": { "value": " Whose " } "entity": " contact " } ] "position": [ { "confidence": 0.98895446748051 "start": 9 "end": 14 "body": " sound " "value": { "value": " sound " } "entity": " position " } ] "intent": [ { "confidence": 0.98819787931465 "value": { "value": " roster_query " } "entity": " intent " } ] } "confidence": null "_text": " Whose on sound tomorrow " "intent": " default_intent " "intent_id": " default_intent_id " } ]

From: Julien Odent [mailto:notifications@github.com] Sent: Tuesday, 9 January 2018 4:15 AM To: wit-ai/wit Cc: bmcclenahan; Mention Subject: Re: [wit-ai/wit] contact entity needs some filtering (#894)

@bmcclenahanhttps://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fbmcclenahan&data=02%7C01%7C%7Ca70ff5fc39514de30ad608d556c3c4cc%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636510321179179924&sdata=ExbpLUmsEThj%2BIP83NWeKeUNZaftT%2FbbRunIzLRYnQk%3D&reserved=0 I feel you. I'll look into improving wit/contact this week. It would help a lot if you can provide a bunch of false positives. Thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fwit-ai%2Fwit%2Fissues%2F894%23issuecomment-356048235&data=02%7C01%7C%7Ca70ff5fc39514de30ad608d556c3c4cc%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636510321179179924&sdata=%2FEM%2BSOi97MkPOjm8OnfZ6UG9jdsvotMyoGjzr6aCbAc%3D&reserved=0, or mute the threadhttps://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAYz9TfmskS19rn6gwJNM8RuvPFouU3sbks5tIlsygaJpZM4RJ0vR&data=02%7C01%7C%7Ca70ff5fc39514de30ad608d556c3c4cc%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636510321179179924&sdata=obsAP%2FYO%2Fcycgw9xGd3XrA5AGV4BLFZKHJ9GLDUTBoE%3D&reserved=0.

patapizza commented 6 years ago

This is now fixed.

bmcclenahan commented 6 years ago

Thanks guys

From: Julien Odent notifications@github.com Sent: Friday, 12 January 2018 6:15 AM To: wit-ai/wit Cc: bmcclenahan; Mention Subject: Re: [wit-ai/wit] contact entity needs some filtering (#894)

This is now fixed.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fwit-ai%2Fwit%2Fissues%2F894%23issuecomment-357048257&data=02%7C01%7C%7C35d629ecce2649f58fbc08d559300f2f%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636512985305799448&sdata=oUVsAunCDgwB3awK%2BVujYFHwU0s9RRViGTi3Me7zTL8%3D&reserved=0, or mute the threadhttps://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAYz9TddzDe5QeXM-2D9pl2TG3FAeTJ0Xks5tJmvfgaJpZM4RJ0vR&data=02%7C01%7C%7C35d629ecce2649f58fbc08d559300f2f%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636512985305799448&sdata=Q8tSrFACfMMNucUyhYhWf8vZiPRSA%2BAmwnCyEZ4mheE%3D&reserved=0.