search

RasaHQ / rasa

💬 Open source machine learning framework to automate text- and voice-based conversations: NLU, dialogue management, connect to Slack, Facebook, and more - Create chatbots and voice assistants
https://rasa.com/docs/rasa/
Apache License 2.0
18.64k stars 4.6k forks source link

Add more training data to moodbot tutorial #1832

Closed thomas-mitchell closed 5 years ago

thomas-mitchell commented 6 years ago

Rasa Core version (e.g. 0.7.3): 0.7.3

Operating system (windows, osx, ...): Windows 10

Issue: Was getting this error when running the Moodbot tutorial:

WARNING:rasa_core.featurizers:Couldn't set most probable feature 'intent_greeting', it wasn't found in the feature list of the domain. Make sure you added all intents and entities to the domain.

Fixed it by changing the nlu.md file "greeting" intent to "greet". The domain and story files refer to the intent as "greet", not "greeting".

Also I had to add extra training examples to the nlu.md file for the training to work correctly, otherwise I was getting the error (which is mentioned in the RASA NLU FAQ):

UndefinedMetricWarning: F-score is ill-defined and being set to 0.0 in labels with no predicted samples.

Before adding extra training examples when I ran the Moodbot in RASA Core it was reading "hello" as the goodbye intent.

mhjb commented 6 years ago

I'm getting similar errors. I changed that intent name to greet (thanks @thomas-mitchell), but still get the below. I'm also wondering why I'm seeing all that DEBUG stuff, which doesn't match the tutorial.

➜  moodbot python -m rasa_core.run -d models/dialogue -u models/nlu/current
INFO:root:Rasa process starting
Using TensorFlow backend.
dyld: warning, LC_RPATH $ORIGIN/../../_solib_darwin_x86_64/_U_S_Stensorflow_Spython_C_Upywrap_Utensorflow_Uinternal.so___Utensorflow in /Users/mhjb/Library/Python/2.7/lib/python/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so being ignored in restricted program because it is a relative path
2017-11-02 21:03:36.055088: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX
DEBUG:pip.utils:lzma module is not available
DEBUG:pip.vcs:Registered VCS backend: git
DEBUG:pip.vcs:Registered VCS backend: hg
DEBUG:pip.vcs:Registered VCS backend: svn
DEBUG:pip.vcs:Registered VCS backend: bzr
INFO:rasa_nlu.components:Added 'nlp_spacy' to component cache. Key 'nlp_spacy-en'.
INFO:root:Finished loading agent, starting input channel & server.
Bot loaded. Type a message and press enter :
hi
DEBUG:rasa_core.tracker_store:Could not find a tracker for id 'default'
DEBUG:rasa_core.processor:Received user message 'hi' with intent '{u'confidence': 0.25596497080939223, u'name': u'goodbye'}' and entities  '[]'
DEBUG:rasa_core.processor:Logged UserUtterance - tracker now has 2 events
DEBUG:rasa_core.processor:Current slot values:

DEBUG:rasa_core.policies.memoization:Current tracker state [
    None
    None
    [(u'intent_goodbye', 1), (u'prev_action_listen', 1)]]
DEBUG:rasa_core.policies.memoization:Used memorised next action '6'
DEBUG:rasa_core.policies.ensemble:Predicted next action #6 with prob 1.00.
Bye
DEBUG:rasa_core.processor:Action 'utter_goodbye' ended with events '[]'
DEBUG:rasa_core.policies.memoization:Current tracker state [
    None
    [(u'intent_goodbye', 1), (u'prev_action_listen', 1)]
    [(u'intent_goodbye', 1), (u'prev_utter_goodbye', 1)]]
DEBUG:rasa_core.policies.memoization:Used memorised next action '0'
DEBUG:rasa_core.policies.ensemble:Predicted next action #0 with prob 1.00.
DEBUG:rasa_core.processor:Action 'action_listen' ended with events '[]'
DEBUG:rasa_core.processor:Current topic: None
andreafspeziale commented 6 years ago

@mhjb @thomas-mitchell Hi I'm experimenting too and this sample has something wrong or (more prob) I'm missing something.

For my understanding the domain.yml - nlu.md - stories.md are referring each others as follow:

Bot loaded. Type a message and press enter :
hey

The "greet intent" of the domain.yml is mapped to the "greet intent" of the nlu.md

Now all the paths described in stories.md (except for the "goodbye") start wi

* _greet

It's detected and the bot answers with utter_greet template described in the domain.md

Hey! How are you?
1: great (great)
2: super sad (super sad)

Now if text I am sad the logs return

DEBUG:rasa_core.processor:Received user message 'I am sad' with intent '{u'confidence': 0.34921424880576996, u'name': u'mood_great'}'

DEBUG:rasa_core.processor:Logged UserUtterance - tracker now has 5 events
DEBUG:rasa_core.processor:Current slot values: 

DEBUG:rasa_core.policies.memoization:Current tracker state [
    [(u'intent_greet', 1), (u'prev_action_listen', 1)]
    [(u'intent_greet', 1), (u'prev_utter_greet', 1)]
    [(u'intent_mood_great', 1), (u'prev_action_listen', 1)]]
DEBUG:rasa_core.policies.memoization:Used memorised next action '5'
DEBUG:rasa_core.policies.ensemble:Predicted next action #5 with prob 1.00.
Great carry on!

This is very strange.

My questions: 1) If my next chat is something negative like "no-never-no way" will the bot improve its understanding that "I am sad" is not "mood_great" 2) Is something missing or my understanding is wrong? 3) DEBUG:rasa_core.tracker_store:Could not find a tracker for id 'default' Is this message because it is the first time I'm asking something? 4) The paths in stories are unique (happy path OR sad path 1 etc.)

kofronpi commented 6 years ago

I followed the tutorial too, and end up with this:

python -m rasa_core.run -d models/dialogue -u models/nlu/current
INFO:root:Rasa process starting
Using TensorFlow backend.
2017-11-02 18:33:48.093995: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
DEBUG:pip.utils:lzma module is not available
DEBUG:pip.vcs:Registered VCS backend: git
DEBUG:pip.vcs:Registered VCS backend: hg
DEBUG:pip.vcs:Registered VCS backend: svn
DEBUG:pip.vcs:Registered VCS backend: bzr
INFO:rasa_nlu.components:Added 'nlp_spacy' to component cache. Key 'nlp_spacy-en'.
INFO:root:Finished loading agent, starting input channel & server.
Bot loaded. Type a message and press enter : 
hello
DEBUG:rasa_core.tracker_store:Could not find a tracker for id 'default'
DEBUG:rasa_core.processor:Received user message 'hello' with intent '{u'confidence': 0.37011225626992345, u'name': u'mood_affirm'}' and entities  '[]'
DEBUG:rasa_core.processor:Logged UserUtterance - tracker now has 2 events
DEBUG:rasa_core.processor:Current slot values: 

DEBUG:rasa_core.policies.memoization:Current tracker state [
    None
    None
    [(u'intent_mood_affirm', 1), (u'prev_action_listen', 1)]]
DEBUG:rasa_core.policies.memoization:Used memorised next action '5'
DEBUG:rasa_core.policies.ensemble:Predicted next action #5 with prob 1.00.
Great carry on!
DEBUG:rasa_core.processor:Action 'utter_happy' ended with events '[]'
DEBUG:rasa_core.policies.memoization:Current tracker state [
    None
    [(u'intent_mood_affirm', 1), (u'prev_action_listen', 1)]
    [(u'intent_mood_affirm', 1), (u'prev_utter_happy', 1)]]
DEBUG:rasa_core.policies.memoization:Used memorised next action '0'
DEBUG:rasa_core.policies.ensemble:Predicted next action #0 with prob 1.00.
DEBUG:rasa_core.processor:Action 'action_listen' ended with events '[]'
DEBUG:rasa_core.processor:Current topic: None

It does not consider "greeting" at all ?

andreafspeziale commented 6 years ago

@kofronpi Is it happening also changing the nlu.md file editing "greeting" intent to "greet"?

kofronpi commented 6 years ago

@andreafspeziale yeah it works a tad better now, but I have a new issue when I say "super sad":

INFO:root:Rasa process starting
Using TensorFlow backend.
2017-11-03 09:49:55.606241: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
DEBUG:pip.utils:lzma module is not available
DEBUG:pip.vcs:Registered VCS backend: git
DEBUG:pip.vcs:Registered VCS backend: hg
DEBUG:pip.vcs:Registered VCS backend: svn
DEBUG:pip.vcs:Registered VCS backend: bzr
INFO:rasa_nlu.components:Added 'nlp_spacy' to component cache. Key 'nlp_spacy-en'.
INFO:root:Finished loading agent, starting input channel & server.
Bot loaded. Type a message and press enter : 
hello
DEBUG:rasa_core.tracker_store:Could not find a tracker for id 'default'
DEBUG:rasa_core.processor:Received user message 'hello' with intent '{u'confidence': 0.26793719919718278, u'name': u'greet'}' and entities  '[]'
DEBUG:rasa_core.processor:Logged UserUtterance - tracker now has 2 events
DEBUG:rasa_core.processor:Current slot values: 

DEBUG:rasa_core.policies.memoization:Current tracker state [
    None
    None
    [(u'intent_greet', 1), (u'prev_action_listen', 1)]]
DEBUG:rasa_core.policies.memoization:Used memorised next action '2'
DEBUG:rasa_core.policies.ensemble:Predicted next action #2 with prob 1.00.
Hey! How are you?
1: great (great)
2: super sad (super sad)
DEBUG:rasa_core.processor:Action 'utter_greet' ended with events '[]'
DEBUG:rasa_core.policies.memoization:Current tracker state [
    None
    [(u'intent_greet', 1), (u'prev_action_listen', 1)]
    [(u'intent_greet', 1), (u'prev_utter_greet', 1)]]
DEBUG:rasa_core.policies.memoization:Used memorised next action '0'
DEBUG:rasa_core.policies.ensemble:Predicted next action #0 with prob 1.00.
DEBUG:rasa_core.processor:Action 'action_listen' ended with events '[]'
DEBUG:rasa_core.processor:Current topic: None
super sad
DEBUG:rasa_core.tracker_store:Recreating tracker for id 'default'
DEBUG:rasa_core.processor:Received user message 'super sad' with intent '{u'confidence': 0.23376077220827138, u'name': u'mood_great'}' and entities  '[]'
DEBUG:rasa_core.processor:Logged UserUtterance - tracker now has 5 events
DEBUG:rasa_core.processor:Current slot values: 

DEBUG:rasa_core.policies.memoization:Current tracker state [
    [(u'intent_greet', 1), (u'prev_action_listen', 1)]
    [(u'intent_greet', 1), (u'prev_utter_greet', 1)]
    [(u'intent_mood_great', 1), (u'prev_action_listen', 1)]]
DEBUG:rasa_core.policies.memoization:Used memorised next action '5'
DEBUG:rasa_core.policies.ensemble:Predicted next action #5 with prob 1.00.
Great carry on!
DEBUG:rasa_core.processor:Action 'utter_happy' ended with events '[]'
DEBUG:rasa_core.policies.memoization:Current tracker state [
    [(u'intent_greet', 1), (u'prev_utter_greet', 1)]
    [(u'intent_mood_great', 1), (u'prev_action_listen', 1)]
    [(u'intent_mood_great', 1), (u'prev_utter_happy', 1)]]
DEBUG:rasa_core.policies.memoization:Used memorised next action '0'
DEBUG:rasa_core.policies.ensemble:Predicted next action #0 with prob 1.00.
DEBUG:rasa_core.processor:Action 'action_listen' ended with events '[]'
DEBUG:rasa_core.processor:Current topic: None
andreafspeziale commented 6 years ago

@kofronpi Me too. Actually "super sad" is not mapped in the nlu.md. But "I am sad" that is mapped under the "mood_unhappy" intent is not even working.

I love this library and I want to learn more and understand what is happening! (:

kofronpi commented 6 years ago

Well I was prototyping to see if I could use it to develop a bot for my company, so I'm not in love yet but it seems promising ;)

auriginal commented 6 years ago

It is strange that it doesn't match intents when given the exact strings, but as the OP says, adding more examples to nlu.md fixes the issue. There must be a minimum amount of data needed to train it

yipcma commented 6 years ago

following the tutorial, hello will only give me Bye.

tmbo commented 6 years ago

@auriginal or @thomas-mitchell do you mind creating a PR with the additional training data you added? I think a couple more training samples would be great to avoid confusion.

chkiriti commented 6 years ago

Why the moodbot is giving wrong results except for the greet intent?

chkiriti commented 6 years ago

Hi Any one is able to run this through facebook chat bot? if yes, let me know what to do after following all the instructions given.

ladak commented 6 years ago

Has anybody fixed the issue? When I type I am sad, rasa_core does not recognize properly the intent.

I am worried to use this library if for something that simple the library can not even map the intent properly.

I am sad DEBUG:rasa_core.tracker_store:Recreating tracker for id 'default' DEBUG:rasa_core.processor:Received user message 'I am sad' with intent '{'name': 'mood_great', 'confidence': 0.41481768345595371}' and entities '[]' DEBUG:rasa_core.processor:Logged UserUtterance - tracker now has 22 events DEBUG:rasa_core.processor:Current slot values:

YuukanOO commented 6 years ago

Using the master branch works for me. It correctly predict my intents.