Reprompt on ambiguous answer

[arekmano]

Description As a user, I'd like the system to ask me for further precision regarding the questioning phase of my case if my answer was unclear or ambiguous.

Scope of Work

• [x] Determine some measures to implement a confidence degree in the answer generation process
• [x] Set a percentage of acceptable confidence ratio through trial and error
• [x] If the answer of the user is too ambiguous ( |Most Sure - 2nd Most Sure| / [1/2(Most Sure + 2nd Most Sure)] ), display a message asking the user to confirm the understanding of the system.
• [x] Create a High-fidelity mock-up of interaction (User inputs his/her problem, system needs clarification due to too low % difference)
• [x] Implement an outlier detection pre-processing step in order to filter out garbage input.

Story Points

• 8

Priority

• High

Risk

• High

Demo requirement

1. The user attempts to trick the system with a misspelling of an important noun or verb relating to a fact i.e. "I am a tanant"
2. The system asks the user to repeat the sentence i.e. "Please repeat" or "Did you mean that you are a tenant?"

Acceptance Criteria

• The system is able to ask users for confirmation to ambiguity of facts or messages
• The users sees the confirmation message
• The user can confirm the message or deny and rewrites a clearer message (with the possibility of the first two steps repeating themselves)

[naregeff]

Verified, high priority.

[Samuel-Campbell]

I suggest making an 'other' category. I'm not sure if RASA takes care of it already though.

[arekmano]

I'm unsure. However, this will most likely need to involve some form of outlier detection, before the phrase is classified. I suggest splitting this story into 2. one for the UI and another for the outlier/anomaly detection method.

Some Reading links: https://medium.com/towards-data-science/a-brief-overview-of-outlier-detection-techniques-1e0b2c19e561 https://en.wikipedia.org/wiki/Anomaly_detection

[ghost]

This is being heavily worked on in junction with #75

[ghost]

Changed to be a combination of percentage difference and threshold.

[arekmano]

Moving to iteration 5 as it will not be closed in iteration 4

[arekmano]

Demo requires beta testers providing answers.

[lancelafontaine]

Functional outlier detection system has been implemented and merged in PR #195, but lacks user input data to our fact's questions to train on. Until this data is obtained, this story will have to remain blocked.