FallbackPolicy shouldn't act on NLU confidence if the last user message was part of a form.

[gzarafbk]

**Rasa Core version**: 0.12.3 **Rasa NLU version**: 0.13.8 **Rasa Core SDK version**: 0.12.1 **Python version**: 3.6.7 **Operating system** (windows, osx, ...): Linux Ubuntu 18.04 **Issue**: Hi, over the last few weeks I have been trying to find a way to find a way to handle generic user messages. In particular, in some specific cases, I would like the system to bypass intent recognition and entity extraction on what the user says, and just go on with the conversation according to the stories. Unfortunately I did not manage that, since the fallback action is always called if no intent is recognized with enough confidence, and the conversation is broken. So far I have temporarily solved the problem by adding some NLU examples of generic user sentences, but this can't really be a solution, since too many examples would be needed, and it would also create confusion when it comes to recognize different intents. Is there a way to just let the user say whatever he likes, possibly just save his sentence in a DB by means of an action and go on with the conversation regardless of the intent and entities? Here follows, for clarity, an example of how I would like my stories to look like: ```markdown ## My story > some_checkpoint - utter_question_1 * generic_user_input - action_save_user_response_in_db - utter_question_2 * generic_user_input - action_save_user_response_in_db ... ``` I have looked around and found something similar in other issues, but I didn't really understand how the solution would work. Thank you in advance for your help, let me know if more information is needed.

[wochinge]

You could use forms to solve this. Maybe this description helps you? Please ask user question in our forum if you don't think it's a bug 🙂

[gzarafbk]

Thank you for your answer,

the form in this solution does actually fill the slot, but the problem is that, once the slot is filled, the fallback action is still called due to zero NLU confidence. Here follows the output of the debug for clarity:

2019-01-05 17:25:07 DEBUG    rasa_core.processor  - Action 'generic_answer_form' ended with events '['SlotSet(key: generic_intent, value: my generic response)', 'Form(None)', 'SlotSet(key: requested_slot, value: None)']'
2019-01-05 17:25:07 DEBUG    rasa_core.policies.fallback  - NLU confidence 0.0 is lower than NLU threshold 0.6. Predicting fallback action: action_fallback

Anything I'm missing? Thanks in advance

PS: sorry for posting in the wrong place, for next time I know ;)

[wochinge]

how do your stories look like?

[gzarafbk]

Here's an example of a part of story where the form is used:

# Manually created story
> aderenza_terapia_farmacologica_precontemplazione_bassa_self_efficacy
    - utter_atf_mot_prec_bas_1
    - generic_answer_form
    - form{"name": "generic_answer_form"}
    - form{"name": null}
    - action_set_db_slot_prec_bas_1
    - action_update_database
    - action_reset_generic_slot
    - utter_atf_mot_prec_bas_2
> asked_atf_mot_prec_bas_2

[wochinge]

So the slot is filled correctly and all actions until utter_atf_mot_prec_bas_2 were executed correctly, right? Then the next action should action_listen. What is the next intent in your stories? Is there enough NLU data there?

I just tried it with this example and it worked as exepected:

## form_story
* greet
  - utter_hello
* greet
  - feedback_form
  - form{"name": "feedback_form"}
  - form{"name": null}
* greet <-- correctly classified by NLU, continue story as expected
  - utter_hello

[gzarafbk]

Actually, the fallback action is called immediately after generic_answer_form fills its slot, so not even action_set_db_sot_prec_bas_1 is performed. The failed NLU recognition (i.e. zero confidence) occurs on the generic sentence itself, the one which is actually used to fill the slot by the form. Wrapping up: 1) the chatbot asks a question to the user (which is utter_atf_mot_prec_bas_1) 2) the form is activated, so the user can answer and his answer can be saved in the slot 3) the user responds with a generic answer (i.e. not necessarily matching any existing intent) 4) the form does its job by filling the slot with the answer of the user 5) NLU fails on the answer of the user independently from the fact that the slot was correctly filled, thus fallback action is called

I'm not sure whether first the slot is filled and then NLU tries to classify the sentence or the other way around. Either way, it goes into fallback.

I hope it's a bit clearer now :)

[wochinge]

the form is activated, so the user can answer and his answer can be saved in the slot

Did the form ask you for the slot to be filled? Could you maybe share the implementation of your form?

The failed NLU recognition (i.e. zero confidence)

Did you connect any NLU model?

[gzarafbk]

Did the form ask you for the slot to be filled?

The form utters utter_ask_generic_intent (which is what i assumed would match utter_ask_problem_message in the example at the link you gave me), if this is what you mean. Then, when the user answers, the fallback action is called and a fallback message is sent by the chatbot. Just to clarify: this happens only if the user's answer doesn't match any intent in the NLU data, otherwise the conversation proceeds correctly.

Could you maybe share the implementation of your form?

Sure, here's the code of the form:

class GenericAnswerFrom(FormAction):

    def name(self):
        return "generic_answer_form"

    @staticmethod
    def required_slots(tracker):
        return["generic_intent"]

    def slot_mappings(self):
        return {"generic_intent": self.from_text()}

    def submit(self, dispatcher, tracker, domain):
        return []

Did you connect any NLU model?

Yes, a NLU model is connected

[wochinge]

Just to clarify: this happens only if the user's answer doesn't match any intent in the NLU data, otherwise the conversation proceeds correctly.

A confidence of 0 is relatively unlikely. However, even without NLU model this should work. Can you please share the log of your action server and your domain file as well?

[gzarafbk]

Sure, here's the action server log up to the point where fallback is called:

STARTING ACTION SERVER
INFO:__main__:Starting action endpoint server...
INFO:rasa_core_sdk.executor:Registered function for 'action_fallback'.
INFO:rasa_core_sdk.executor:Registered function for 'action_retrieve_and_set_user_id'.
INFO:rasa_core_sdk.executor:Registered function for 'action_retrieve_and_set_status_change_score'.
INFO:rasa_core_sdk.executor:Registered function for 'action_retrieve_and_set_self_efficacy_score'.
INFO:rasa_core_sdk.executor:Registered function for 'action_number_out_of_range'.
INFO:rasa_core_sdk.executor:Registered function for 'action_reset_health_literacy_slot'.
INFO:rasa_core_sdk.executor:Registered function for 'action_update_database'.
INFO:rasa_core_sdk.executor:Registered function for 'action_reset_slots'.
INFO:rasa_core_sdk.executor:Registered function for 'action_set_db_slot_intervention_area'.
INFO:rasa_core_sdk.executor:Registered function for 'action_set_db_slot_change_score'.
INFO:rasa_core_sdk.executor:Registered function for 'action_set_db_slot_hl_atf_1'.
INFO:rasa_core_sdk.executor:Registered function for 'action_set_db_slot_hl_atf_2'.
INFO:rasa_core_sdk.executor:Registered function for 'action_set_db_slot_self_efficacy'.
INFO:rasa_core_sdk.executor:Registered function for 'action_set_db_slot_prec_bas_1'.
INFO:rasa_core_sdk.executor:Registered function for 'action_set_db_slot_prec_bas_2'.
INFO:rasa_core_sdk.executor:Registered function for 'action_set_db_slot_prec_bas_3'.
INFO:rasa_core_sdk.executor:Registered function for 'action_set_db_slot_prec_alt_1'.
INFO:rasa_core_sdk.executor:Registered function for 'action_set_db_slot_prec_alt_2'.
INFO:rasa_core_sdk.executor:Registered function for 'action_set_db_slot_cont_bas_1'.
INFO:rasa_core_sdk.executor:Registered function for 'action_set_db_slot_cont_bas_2'.
INFO:rasa_core_sdk.executor:Registered function for 'action_set_db_slot_cont_bas_3'.
INFO:rasa_core_sdk.executor:Registered function for 'action_set_db_slot_cont_alt_1'.
INFO:rasa_core_sdk.executor:Registered function for 'action_set_db_slot_cont_alt_2'.
INFO:rasa_core_sdk.executor:Registered function for 'action_set_profiled'.
INFO:rasa_core_sdk.executor:Registered function for 'action_check_if_profiled'.
INFO:rasa_core_sdk.executor:Registered function for 'action_retrieve_user_slots'.
INFO:rasa_core_sdk.executor:Registered function for 'action_set_low_change_score'.
INFO:rasa_core_sdk.executor:Registered function for 'action_set_medium_change_score'.
INFO:rasa_core_sdk.executor:Registered function for 'action_set_high_change_score'.
INFO:rasa_core_sdk.executor:Registered function for 'action_reset_generic_slot'.
INFO:rasa_core_sdk.executor:Registered function for 'alimentazione_form'.
INFO:rasa_core_sdk.executor:Registered function for 'attivita_fisica_form'.
INFO:rasa_core_sdk.executor:Registered function for 'controllo_glicemia_form'.
INFO:rasa_core_sdk.executor:Registered function for 'intervention_area_form'.
INFO:rasa_core_sdk.executor:Registered function for 'status_change_score_form'.
INFO:rasa_core_sdk.executor:Registered function for 'answer_health_literacy_form'.
INFO:rasa_core_sdk.executor:Registered function for 'generic_answer_form'.
INFO:__main__:Action endpoint is up and running. on ('0.0.0.0', 5055)
DEBUG:rasa_core_sdk.executor:Received request to run 'action_reset_slots'
DEBUG:rasa_core_sdk.executor:Successfully ran 'action_reset_slots'
127.0.0.1 - - [2019-01-08 11:34:30] "POST /webhook HTTP/1.1" 200 209 0.001356
DEBUG:rasa_core_sdk.executor:Received request to run 'action_retrieve_and_set_user_id'
DEBUG:rasa_core_sdk.executor:Successfully ran 'action_retrieve_and_set_user_id'
127.0.0.1 - - [2019-01-08 11:34:30] "POST /webhook HTTP/1.1" 200 244 0.033797
DEBUG:rasa_core_sdk.executor:Received request to run 'action_check_if_profiled'
DEBUG:rasa_core_sdk.executor:Successfully ran 'action_check_if_profiled'
127.0.0.1 - - [2019-01-08 11:34:30] "POST /webhook HTTP/1.1" 200 239 0.001241
DEBUG:rasa_core_sdk.executor:Received request to run 'intervention_area_form'
DEBUG:rasa_core_sdk.forms:There is no active form
DEBUG:rasa_core_sdk.forms:Activated the form 'intervention_area_form'
DEBUG:rasa_core_sdk.forms:Skipping validation
DEBUG:rasa_core_sdk.forms:Request next slot 'area_intervento'
DEBUG:rasa_core_sdk.executor:Successfully ran 'intervention_area_form'
127.0.0.1 - - [2019-01-08 11:34:32] "POST /webhook HTTP/1.1" 200 359 0.001384
DEBUG:rasa_core_sdk.executor:Received request to run 'intervention_area_form'
DEBUG:rasa_core_sdk.forms:The form '{'name': 'intervention_area_form', 'validate': True, 'rejected': False}' is active
DEBUG:rasa_core_sdk.forms:Validating user input '{'intent': {'name': 'scegli_area_intervento', 'confidence': 0.9283026456832886}, 'entities': [{'start': 0, 'end': 30, 'value': 'aderenza terapia farmacologica', 'entity': 'area_intervento', 'confidence': 0.9062406353912459, 'extractor': 'ner_crf'}], 'intent_ranking': [{'name': 'scegli_area_intervento', 'confidence': 0.9283026456832886}, {'name': 'inizia', 'confidence': 0.204351007938385}, {'name': 'interazione_generica', 'confidence': 0.0}, {'name': 'input_numerico', 'confidence': 0.0}, {'name': 'rispondi_health_literacy', 'confidence': 0.0}, {'name': 'comunica_self_efficacy', 'confidence': 0.0}, {'name': 'afferma', 'confidence': 0.0}, {'name': 'nega', 'confidence': 0.0}], 'text': 'aderenza terapia farmacologica'}'
THIS Slot to fill: aderenza terapia farmacologica
THIS Slot values: {}
DEBUG:rasa_core_sdk.forms:Trying to extract requested slot 'area_intervento' ...
DEBUG:rasa_core_sdk.forms:Got mapping '{'type': 'from_entity', 'entity': 'area_intervento', 'intent': ['scegli_area_intervento'], 'not_intent': []}'
DEBUG:rasa_core_sdk.forms:Successfully extracted 'aderenza terapia farmacologica' for requested slot 'area_intervento'
value is aderenza terapia farmacologica
DEBUG:rasa_core_sdk.forms:No slots left to request
DEBUG:rasa_core_sdk.forms:Deactivating the form 'intervention_area_form'
DEBUG:rasa_core_sdk.executor:Successfully ran 'intervention_area_form'
127.0.0.1 - - [2019-01-08 11:34:35] "POST /webhook HTTP/1.1" 200 386 0.001447
DEBUG:rasa_core_sdk.executor:Received request to run 'action_set_db_slot_intervention_area'
DEBUG:rasa_core_sdk.executor:Successfully ran 'action_set_db_slot_intervention_area'
127.0.0.1 - - [2019-01-08 11:34:35] "POST /webhook HTTP/1.1" 200 248 0.000730
DEBUG:rasa_core_sdk.executor:Received request to run 'action_update_database'
DEBUG:rasa_core_sdk.executor:Successfully ran 'action_update_database'
127.0.0.1 - - [2019-01-08 11:34:35] "POST /webhook HTTP/1.1" 200 169 0.038284
DEBUG:rasa_core_sdk.executor:Received request to run 'action_set_db_slot_change_score'
DEBUG:rasa_core_sdk.executor:Successfully ran 'action_set_db_slot_change_score'
127.0.0.1 - - [2019-01-08 11:34:35] "POST /webhook HTTP/1.1" 200 243 0.000760
DEBUG:rasa_core_sdk.executor:Received request to run 'action_set_low_change_score'
DEBUG:rasa_core_sdk.executor:Successfully ran 'action_set_low_change_score'
127.0.0.1 - - [2019-01-08 11:34:36] "POST /webhook HTTP/1.1" 200 246 0.037564
DEBUG:rasa_core_sdk.executor:Received request to run 'action_set_profiled'
DEBUG:rasa_core_sdk.executor:Successfully ran 'action_set_profiled'
127.0.0.1 - - [2019-01-08 11:34:36] "POST /webhook HTTP/1.1" 200 169 0.008463
DEBUG:rasa_core_sdk.executor:Received request to run 'action_set_db_slot_self_efficacy'
DEBUG:rasa_core_sdk.executor:Successfully ran 'action_set_db_slot_self_efficacy'
127.0.0.1 - - [2019-01-08 11:34:36] "POST /webhook HTTP/1.1" 200 244 0.000772
DEBUG:rasa_core_sdk.executor:Received request to run 'action_update_database'
DEBUG:rasa_core_sdk.executor:Successfully ran 'action_update_database'
127.0.0.1 - - [2019-01-08 11:34:39] "POST /webhook HTTP/1.1" 200 169 0.011455
DEBUG:rasa_core_sdk.executor:Received request to run 'action_retrieve_and_set_self_efficacy_score'
DEBUG:rasa_core_sdk.executor:Successfully ran 'action_retrieve_and_set_self_efficacy_score'
127.0.0.1 - - [2019-01-08 11:34:39] "POST /webhook HTTP/1.1" 200 248 0.001179
DEBUG:rasa_core_sdk.executor:Received request to run 'generic_answer_form'
DEBUG:rasa_core_sdk.forms:There is no active form
DEBUG:rasa_core_sdk.forms:Activated the form 'generic_answer_form'
DEBUG:rasa_core_sdk.forms:Skipping validation
DEBUG:rasa_core_sdk.forms:Request next slot 'generic_intent'
DEBUG:rasa_core_sdk.executor:Successfully ran 'generic_answer_form'
127.0.0.1 - - [2019-01-08 11:34:39] "POST /webhook HTTP/1.1" 200 354 0.002066
DEBUG:rasa_core_sdk.executor:Received request to run 'generic_answer_form'
DEBUG:rasa_core_sdk.forms:The form '{'name': 'generic_answer_form', 'validate': True, 'rejected': False}' is active
DEBUG:rasa_core_sdk.forms:Validating user input '{'intent': {'name': None, 'confidence': 0.0}, 'entities': [], 'intent_ranking': [], 'text': 'risposta generica'}'
DEBUG:rasa_core_sdk.forms:Trying to extract requested slot 'generic_intent' ...
DEBUG:rasa_core_sdk.forms:Got mapping '{'type': 'from_text', 'intent': [], 'not_intent': []}'
DEBUG:rasa_core_sdk.forms:Successfully extracted 'risposta generica' for requested slot 'generic_intent'
DEBUG:rasa_core_sdk.forms:No slots left to request
DEBUG:rasa_core_sdk.forms:Deactivating the form 'generic_answer_form'
DEBUG:rasa_core_sdk.executor:Successfully ran 'generic_answer_form'
127.0.0.1 - - [2019-01-08 11:34:47] "POST /webhook HTTP/1.1" 200 372 0.002601
DEBUG:rasa_core_sdk.executor:Received request to run 'action_fallback'
DEBUG:rasa_core_sdk.executor:Successfully ran 'action_fallback'
127.0.0.1 - - [2019-01-08 11:34:47] "POST /webhook HTTP/1.1" 200 300 0.001002

And here's my domain file:

##### SLOTS #####
slots:
  user_id:
    type: unfeaturized
  bot_name:
    type: unfeaturized
    initial_value: "Lisa"
  area_intervento:
    type: dialogue.dialogue_custom_slots.AreaInterventoSlot
  status_change_score:
    type: dialogue.dialogue_custom_slots.StatusChangeScoreSlot
  self_efficacy_score:
    type: dialogue.dialogue_custom_slots.SelfEfficacyScoreSlot
  risposta_health_literacy:
    type: dialogue.dialogue_custom_slots.RispostaHealthLiteracySlot
  db_slot:
    type: unfeaturized
  is_profiled:
    type: dialogue.dialogue_custom_slots.IsProfiledSlot
  generic_intent:
    type: unfeaturized

##### ENTITIES #####
entities:
  - numero
  - area_intervento
  - valore_self_efficacy
  - risposta_health_literacy

##### INTENTS #####
intents:
  - inizia
  - afferma
  - nega
  - input_numerico
  - scegli_area_intervento
  - comunica_self_efficacy
  - rispondi_health_literacy
  - interazione_generica
  - None

##### TEMPLATES #####
templates:

    utter_benvenuto:
      - "Ciao, il mio nome è {bot_name}, un'assistente virtuale che ti aiuterà a conoscere e gestire meglio il diabete, migliorando il tuo stile di vita"

    utter_chiedi_se_pronto:
      - "Per farlo, però, ho bisogno di conoscerti meglio. Sei pronto?"

    utter_perfetto:
      - "Perfetto, cominciamo!"
      - "Ottimo, cominciamo!"
      - "Fantastico, cominciamo!"

    utter_allora_ciao:
      - "Sarà per la prossima volta, a presto!"

    utter_ringrazia:
      - "Grazie"

    utter_fallback:
      - "Risposta non valida, per favore riprova"

    utter_valore_fuori_range:
      - "Il numero inserito è al di fuori dell'intervallo previsto, per favore riprova"

    utter_chiedi_area_intervento:
      - "Scegli quale di queste quattro aree comportamentali ti piacerebbe migliorare il tuo stile di vita: aderenza terapia farmacologica, alimentazione, attività fisica, controllo glicemia"

    utter_chiedi_livello_motivazione:
      - "Indica su scala 1-10 quanto ti senti in grado di cambiare il tuo comportamento in quest'area"

    utter_chiedi_self_efficacy:
      - "Quanto ti senti in grado di migliorare nell'area {area_intervento}, poco o molto?"

    utter_chiedi_atf_1:
      - "Un alto livello di zucchero nel sangue può essere causato da troppa insulina?"

    utter_chiedi_atf_2:
      - "Se ti fai l'insulina la mattina ma salti la colazione, la tua glicemia scenderà?"

    utter_chiedi_al_1:
      - "La dieta per persone diabetiche è una dieta salutare in generale?"

    utter_chiedi_al_2:
      - "Una porzione di pollo ha più carboidrati di una porzione di patate?"

    utter_chiedi_al_3:
      - "Il succo d'arancia ha più grassi del latte?"

    utter_chiedi_al_4:
      - "Il succo di frutta senza zuccheri alza il livello di glicemia?"

    utter_chiedi_al_5:
      - "Cucinare con l'olio d'oliva può prevenire l'aumento di colesterolo?"

    utter_chiedi_af_1:
      - "Fare regolarmente esercizio fisico può aiutare a ridurre il rischio di pressione alta?"

    utter_chiedi_af_2:
      - "Per una persona con un buon controllo glicemico l'esercizio fisico non ha effetto sul livello di zucchero nel sangue?"

    utter_chiedi_cg_1:
      - "L'emoglobina glicosilata è un test che misura la media dello zucchero nel sangue?"

    utter_chiedi_cg_2:
      - "Un'infenzione può causare un aumento della glicemia?"

    utter_chiedi_cg_3:
      - "Il test delle urine e l'analisi del sangue vanno bene entrambi per misurare il livello di zuccheri nel sangue?"

    utter_chiedi_cg_4:
      - "Quando hai l'influenza dovresti misurarti la glicemia più spesso?"

    utter_risposta_bassa_motivazione:
      - "Non molto alta, vedo..."

    utter_risposta_media_motivazione:
      - "Vedo che ci stai pensando..."

    utter_risposta_alta_motivazione:
      - "Ottimo, vedo che la tua motivazione è alta!"

    utter_introduci_health_literacy:
      - "Ora se per te va bene ti porrò alcune semplici domande. Cominciamo!"

    utter_atf_mot_prec_bas_1:
      - "Prendere le medicine o l'insulina come da prescrizione del tuo medico aiuta a mantenersi in salute, ad evitare rischi... quali ragioni di solito te lo impediscono?"

    utter_atf_mot_prec_bas_2:
      - "Capisco... ci sono delle cose che vorresti cambiare in particolare?"

    utter_atf_mot_prec_bas_3:
      - "Grazie per aver chiarito questi aspetti... considera che anche dei piccoli cambiamenti graduali nel tempo possono fare la differenza e richiedono poco sforzo, non trovi?"

    utter_atf_mot_prec_bas_4:
      - "Se ti va ne riparliamo la prossima settimana... ora ti chiedo solo di compilare un breve questionario sull'esperienza di parlare con me oggi che trovi a questo link ...alla prossima! https://goo.gl/forms/ox0Y5X7Pmk8kBdvz2"

    utter_atf_mot_prec_alt_1:
      - "Bene... prendere le medicine o l'insulina come da prescrizione del tuo medico aiuta a mantenersi in salute ed evitare rischi... quali ragioni di solito te lo impediscono?"

    utter_atf_mot_prec_alt_2:
      - "Capisco... ci sono delle cose che vorresti cambiare in particolare?"

    utter_atf_mot_prec_alt_3:
      - "Grazie per aver chiarito questi aspetti, se vuoi ne riparliamo la prossima settimana... ora ti chiedo solo di compilare un breve questionario (link sotto) sull'esperienza di parlare con me ...alla prossima! https://goo.gl/forms/ox0Y5X7Pmk8kBdvz2"

    utter_atf_mot_cont_bas_1:
      - "Prendere le medicine o l'insulina come da prescrizione medica è la scelta migliore per la tua salute. Spesso però prima di decidere valutiamo costi e benefici: e per te quali sono i benefici?"

    utter_atf_mot_cont_bas_2:
      - "E quali sono i costi di prendere una nuova medicina per te?"

    utter_atf_mot_cont_bas_3:
      - "Grazie per avermi chiarito questi aspetti... considera però che anche dei piccoli cambiamenti graduali nel tempo possono fare la differenza, e richiedono poco sforzo! Sei d'accordo?"

    utter_atf_mot_cont_bas_4:
      - "Se vuoi ne riparliamo la prossima settimana... ora ti chiedo solo di compilare un breve questionario (link sotto) sull'esperienza di parlare oggi con me... alla possima! https://goo.gl/forms/ox0Y5X7Pmk8kBdvz2"

    utter_atf_mot_cont_alt_1:
      - "Prendere le medicine o l'insulina come da prescrizione medica è la scelta migliore per la tua salute. Spesso però prima di decidere valutiamo costi e benefici: e per te quali sono i benefici?"

    utter_atf_mot_cont_alt_2:
      - "E quali sono i costi di prendere una nuova medicina per te?"

    utter_atf_mot_cont_alt_3:
      - "Grazie per aver chiarito questi aspetti, se vuoi ne riparliamo la prossima settimana... ora ti chiedo solo di compilare un breve questionario (link sotto) sull'esperienza di parlare con me! Alla prossima! https://goo.gl/forms/ox0Y5X7Pmk8kBdvz2"

    utter_risposta_esatta:
      - "Risposta esatta!"

    utter_risposta_falsa_errata:
      - "Mmh... no è vero, guarda questo video e dimmi poi cosa ne pensi!"

    utter_risposta_vera_errata:
      - "Mmh... no è falso, prova a guardare questo video e dimmi cosa ne pensi!"

    utter_link_video_atf_prep_insulina_glicemia:
      - "https://www.youtube.com/watch?v=qsYtahJg2PY"

    utter_link_video_atf_prep_insulina_organismo_umano:
      - "https://www.youtube.com/watch?v=qzhRZ7GuZvY"

    utter_atf_mot_grazie_risposte:
      - "Grazie per le tue risposte... riparliamo di questo argomento presto... ora ti chiedo solo di compilare un breve questionario (link sotto) sull'esperienza di parlare con me oggi. Alla prossima! https://goo.gl/forms/ox0Y5X7Pmk8kBdvz2"

    utter_already_profiled:
      - "Mi risulta che tu sia già stato profilato. Riprendiamo da subito dopo la tua profilazione!"

    utter_chiedi_livello_motivazione_1:
      - "Da 1-10 la tua motivazione è maggiore di 4?"

    utter_chiedi_livello_motivazione_2:
      - "E di 8?"

    utter_ask_generic_intent:
      - "Fornisci una risposta generica"

##### ACTIONS #####
actions:
  - utter_benvenuto
  - utter_chiedi_se_pronto
  - utter_perfetto
  - utter_allora_ciao
  - utter_ringrazia
  - utter_fallback
  - utter_valore_fuori_range
  - utter_chiedi_area_intervento
  - utter_chiedi_livello_motivazione
  - utter_chiedi_self_efficacy
  - utter_chiedi_atf_1
  - utter_chiedi_atf_2
  - utter_chiedi_al_1
  - utter_chiedi_al_2
  - utter_chiedi_al_3
  - utter_chiedi_al_4
  - utter_chiedi_al_5
  - utter_chiedi_af_1
  - utter_chiedi_af_2
  - utter_chiedi_cg_1
  - utter_chiedi_cg_2
  - utter_chiedi_cg_3
  - utter_chiedi_cg_4
  - utter_risposta_bassa_motivazione
  - utter_risposta_media_motivazione
  - utter_risposta_alta_motivazione
  - utter_introduci_health_literacy
  - utter_atf_mot_prec_bas_1
  - utter_atf_mot_prec_bas_2
  - utter_atf_mot_prec_bas_3
  - utter_atf_mot_prec_bas_4
  - utter_atf_mot_prec_alt_1
  - utter_atf_mot_prec_alt_2
  - utter_atf_mot_prec_alt_3
  - utter_atf_mot_cont_bas_1
  - utter_atf_mot_cont_bas_2
  - utter_atf_mot_cont_bas_3
  - utter_atf_mot_cont_bas_4
  - utter_atf_mot_cont_alt_1
  - utter_atf_mot_cont_alt_2
  - utter_atf_mot_cont_alt_3
  - utter_risposta_esatta
  - utter_risposta_falsa_errata
  - utter_risposta_vera_errata
  - utter_link_video_atf_prep_insulina_glicemia
  - utter_link_video_atf_prep_insulina_organismo_umano
  - utter_atf_mot_grazie_risposte
  - utter_already_profiled
  - utter_chiedi_livello_motivazione_1
  - utter_chiedi_livello_motivazione_2
  - utter_ask_generic_intent
  - action_fallback
  - action_retrieve_and_set_user_id
  - action_retrieve_and_set_status_change_score
  - action_retrieve_and_set_self_efficacy_score
  - action_number_out_of_range
  - action_reset_health_literacy_slot
  - action_update_database
  - action_reset_slots
  - action_set_db_slot_intervention_area
  - action_set_db_slot_change_score
  - action_set_db_slot_hl_atf_1
  - action_set_db_slot_hl_atf_2
  - action_set_db_slot_self_efficacy
  - action_set_db_slot_prec_bas_1
  - action_set_db_slot_prec_bas_2
  - action_set_db_slot_prec_bas_3
  - action_set_db_slot_prec_alt_1
  - action_set_db_slot_prec_alt_2
  - action_set_db_slot_cont_bas_1
  - action_set_db_slot_cont_bas_2
  - action_set_db_slot_cont_bas_3
  - action_set_db_slot_cont_alt_1
  - action_set_db_slot_cont_alt_2
  - action_set_profiled
  - action_check_if_profiled
  - action_retrieve_user_slots
  - action_set_low_change_score
  - action_set_medium_change_score
  - action_set_high_change_score
  - action_reset_generic_slot

##### FORMS #####
forms:
  - answer_health_literacy_form
  - intervention_area_form
  - status_change_score_form
  - generic_answer_form

[wochinge]

Ok, this looks great :+1:

Can you also attach the story for > asked_atf_mot_prec_bas_2? Seems that it is failing here.

[gzarafbk]

I just realized that after my last modifications to the chatbot I actually no longer need that checkpoint (in the sense that it just had one possible way to proceed, therefore i can remove the checkpoint). So the story actually looks like this:

# Manually created story
> aderenza_terapia_farmacologica_precontemplazione_bassa_self_efficacy
    - utter_atf_mot_prec_bas_1
    - generic_answer_form
    - form{"name": "generic_answer_form"}
    - form{"name": null}
    - action_set_db_slot_prec_bas_1
    - action_update_database
    - action_reset_generic_slot
    - utter_atf_mot_prec_bas_2
    - generic_answer_form
    - form{"name": "generic_answer_form"}
    - form{"name": null}
    - action_set_db_slot_prec_bas_2
    - action_update_database
    - action_reset_generic_slot
    - utter_atf_mot_prec_bas_3
    - generic_answer_form
    - form{"name": "generic_answer_form"}
    - form{"name": null}
    - action_set_db_slot_prec_bas_3
    - action_update_database
    - action_reset_generic_slot
    - utter_atf_mot_prec_bas_4
    - action_restart

The logic is pretty much the same (indeed i re-trained and re-tested, but ended up with exactly the same problem)

[wochinge]

And what's your core policy configuration?

[gzarafbk]

My policy configuration file contains this:

policies:
  - name: KerasPolicy
    epochs: 5
    max_history: 30
  - name: FallbackPolicy
    fallback_action_name: 'action_fallback'
    nlu_threshold: 0.6
    core_threshold: 0.6
  - name: MemoizationPolicy
    max_history: 50
  - name: FormPolicy

Was this the question?

[wochinge]

Could you please try with

policies:
  - name: KerasPolicy
    epochs: 5
    max_history: 30
  - name: MemoizationPolicy
    max_history: 50
  - name: FallbackPolicy
    fallback_action_name: 'action_fallback'
    nlu_threshold: 0.6
    core_threshold: 0.6
  - name: FormPolicy

[gzarafbk]

I have. Unfortunately I end up with the same problem: the slot is filled, but if the intent is classified as "None" with 0.0 confidence, fallback action is called.

[wochinge]

Mhm, this is really strange. Could you maybe run core with the --debu and post the core log file as well?

[gzarafbk]

Sure, I pasted it into a file for simplicity: core_run.txt. It is up to the point where fallback is called (sorry if it's messy and in Italian, but it should contain everything). Let me know if anything else is needed, and thanks again.

[akelad]

@gzarafbk are you using the tensorflow_embedding pipeline for NLU? The intent being classified as "None" is usually caused by the sentence not containing any words that are in your vocabulary. And you can handle this by adding OOV handling to the tensorflow pipeline: https://rasa.com/docs/nlu/components/#intent-featurizer-count-vectors

[wochinge]

@akelad I think this is actually a bug. The fallback policy checks for the last user message which in this case was part of the form (https://github.com/RasaHQ/rasa_core/blob/b65d6716b4477e4ca023f24c03033f7fedd0f4ce/rasa_core/policies/fallback.py#L86). I think it should only check the nlu confidence if the last event was a user message, right (considering some slotsetting events afterwards)

[gzarafbk]

@akelad I have tried following your suggestion, but nothing seems to change. This is the pipeline, am I doing it right?

pipeline:
  - name: "tokenizer_whitespace"
  - name: "ner_crf"
  - name: "intent_featurizer_count_vectors"
  - name: "intent_classifier_tensorflow_embedding"
    "OOV_token": "si"

I have used "si" as OOV_token, which matches the existing intent afferma in my nlu file. Should I use a specific different one?

[akelad]

pipeline:
  - name: "tokenizer_whitespace"
  - name: "ner_crf"
  - name: "intent_featurizer_count_vectors"
    OOV_token: si
  - name: "intent_classifier_tensorflow_embedding"

This is the way to do it, can you try that please? Also, I'd recommend not using "si", but rather something like "oov" and then adding some training examples to intents where it's likely that an oov word will come (for example, the out_of_scope intent). Let me know if it works with this configuration

[wochinge]

@akelad This is indeed a bug. Consider this story:

## story 123
* greet
  - feedback_form
  - form{"name": "feedback_form"}
 --> user message to fill form slot <-- classified with confidence < threshold
  - form{"name": null}
  - utter_finished_form <-- instead of this, the fallback action is called since the last user message was beyond the threshold

[akelad]

@wochinge yes, that part is, but the oov thing is still something that should be added to the pipeline in general ;)

[akelad]

Just for your clarification @gzarafbk, you should try the NLU config I showed you and it will fix your intent being classified as "None" issue. However you did uncover a bug that the fallback policy shouldn't pay attention to intent confidence within a form, which we will look into and fix

[wochinge]

A suggestion to fix this bug is to check whether the last executed action was ACTION_LISTEN before checking the NLU confidence of the last message.

[Ghostvv]

@gzarafbk could you please try fix-fallback branch of rasa_core with your form and original config?