My understanding: When creating a TimeLogicAdapter with no additional parameters, the default positive and negative lists are applied. When hitting the positive list, confidence level is expeted to be set to 1, when hitting the negative list, level will be 0.
From the 4 examples below, the TimeLogicAdapter two times incorrectly indicates confidence level 1.
Question 1: 'What is 4 + 9?'
TimeLogicAdapter returns 1.
Question 2: 'What time is it?'
TimeLogicAdapter returns 1.
Question 3: 'What is AI?'
TimeLogicAdapter returns 1.
Question 4: 'What is your favorite number?'
TimeLogicAdapter returns 0.
While 2 and 4 the Adapter returns the correct confidence level, on question 1 and 3 returning 1 while not hitting the Adapter's default positive list, to me, seems to unexpected behaviour and in case of 3 leading to a incorrect answer.
Also, I'm wondering why the Adapter returned 0 in question 4 while it returns 1 in question 3.
Did i miss something or is this unexpected and inconsistent behaviour?
Chatbot Configuration
import spacy
import logging
from chatterbot import ChatBot
from chatterbot.trainers import ChatterBotCorpusTrainer
logging.basicConfig(level=logging.INFO)
nlp = spacy.load('en_core_web_sm')
english_bot = ChatBot(
"Luke",
storage_adapter="chatterbot.storage.SQLStorageAdapter",
logic_adapters=[
"chatterbot.logic.MathematicalEvaluation",
"chatterbot.logic.TimeLogicAdapter",
"chatterbot.logic.BestMatch"
],
read_only=True)
trainer = ChatterBotCorpusTrainer(english_bot)
trainer.train("chatterbot.corpus.english")
# Print an example of getting one math based response
response = english_bot.get_response('What is 4 + 9?')
print(response)
# Print an example of getting one time based response
response = english_bot.get_response('What time is it?')
print(response)
# Print an example of getting one bestmatch response
response = english_bot.get_response('What is AI?')
print(response)
# Print an example of getting one bestmatch response
response = english_bot.get_response('What is your favorite number?')
print(response)
Logs
Training ai.yml: [####################] 100%
Training botprofile.yml: [####################] 100%
Training computers.yml: [####################] 100%
Training conversations.yml: [####################] 100%
Training emotion.yml: [####################] 100%
Training food.yml: [####################] 100%
Training gossip.yml: [####################] 100%
Training greetings.yml: [####################] 100%
Training health.yml: [####################] 100%
Training history.yml: [####################] 100%
Training humor.yml: [####################] 100%
Training literature.yml: [####################] 100%
Training money.yml: [####################] 100%
Training movies.yml: [####################] 100%
Training politics.yml: [####################] 100%
Training psychology.yml: [####################] 100%
Training science.yml: [####################] 100%
Training sports.yml: [####################] 100%
Training trivia.yml: [####################] 100%
INFO:chatterbot.chatterbot:MathematicalEvaluation selected "4 + 9 = 13" as a response with a confidence of 1
INFO:chatterbot.chatterbot:TimeLogicAdapter selected "The current time is 11:20 PM" as a response with a confidence of 1
INFO:chatterbot.chatterbot:Beginning search for close text match
INFO:chatterbot.chatterbot:Processing search results
INFO:chatterbot.chatterbot:Similar text found: What is AI? 0.72
INFO:chatterbot.chatterbot:Using "What is AI?" as a close match to "What is 4 + 9?" with a confidence of 0.72
4 + 9 = 13
INFO:chatterbot.chatterbot:Selecting response from 12 optimal responses.
INFO:chatterbot.response_selection:Selecting first response from list of 12 options.
INFO:chatterbot.chatterbot:Response selected. Using "Artificial Intelligence is the branch of engineering and science devoted to constructing machines that think."
INFO:chatterbot.chatterbot:BestMatch selected "Artificial Intelligence is the branch of engineering and science devoted to constructing machines that think." as a response with a confidence of 0.72
INFO:chatterbot.chatterbot:Not processing the statement using MathematicalEvaluation
INFO:chatterbot.chatterbot:TimeLogicAdapter selected "The current time is 11:20 PM" as a response with a confidence of 1
INFO:chatterbot.chatterbot:Beginning search for close text match
INFO:chatterbot.chatterbot:Processing search results
INFO:chatterbot.chatterbot:Similar text found: Are you sentient? 0.42
INFO:chatterbot.chatterbot:Similar text found: What is your business 0.49
INFO:chatterbot.chatterbot:Similar text found: What is your idea 0.55
INFO:chatterbot.chatterbot:Similar text found: What are you then? 0.59
INFO:chatterbot.chatterbot:Similar text found: What is it like? 0.69
INFO:chatterbot.chatterbot:Using "Are you sentient?" as a close match to "What time is it?" with a confidence of 0.42
The current time is 11:20 PM
INFO:chatterbot.chatterbot:Selecting response from 18 optimal responses.
INFO:chatterbot.response_selection:Selecting first response from list of 18 options.
INFO:chatterbot.chatterbot:Response selected. Using "Sort of."
INFO:chatterbot.chatterbot:BestMatch selected "Sort of." as a response with a confidence of 0.42
INFO:chatterbot.chatterbot:Not processing the statement using MathematicalEvaluation
INFO:chatterbot.chatterbot:TimeLogicAdapter selected "The current time is 11:20 PM" as a response with a confidence of 1
INFO:chatterbot.chatterbot:Beginning search for close text match
INFO:chatterbot.chatterbot:Processing search results
INFO:chatterbot.chatterbot:Similar text found: What is AI? 1.0
INFO:chatterbot.chatterbot:Using "What is AI?" as a close match to "What is AI?" with a confidence of 1.0
The current time is 11:20 PM
INFO:chatterbot.chatterbot:Selecting response from 12 optimal responses.
INFO:chatterbot.response_selection:Selecting first response from list of 12 options.
INFO:chatterbot.chatterbot:Response selected. Using "Artificial Intelligence is the branch of engineering and science devoted to constructing machines that think."
INFO:chatterbot.chatterbot:BestMatch selected "Artificial Intelligence is the branch of engineering and science devoted to constructing machines that think." as a response with a confidence of 1.0
INFO:chatterbot.chatterbot:Not processing the statement using MathematicalEvaluation
INFO:chatterbot.chatterbot:TimeLogicAdapter selected "The current time is 11:20 PM" as a response with a confidence of 0
INFO:chatterbot.chatterbot:Beginning search for close text match
INFO:chatterbot.chatterbot:Processing search results
INFO:chatterbot.chatterbot:Similar text found: What is your favorite number 0.98
INFO:chatterbot.chatterbot:Using "What is your favorite number" as a close match to "What is your favorite number?" with a confidence of 0.98
I find I'm quite fond of the number 42.
INFO:chatterbot.chatterbot:Selecting response from 6 optimal responses.
INFO:chatterbot.response_selection:Selecting first response from list of 6 options.
INFO:chatterbot.chatterbot:Response selected. Using "I find I'm quite fond of the number 42."
INFO:chatterbot.chatterbot:BestMatch selected "I find I'm quite fond of the number 42." as a response with a confidence of 0.98
Issue: Unexpected return of confidence value 1
My understanding: When creating a TimeLogicAdapter with no additional parameters, the default positive and negative lists are applied. When hitting the positive list, confidence level is expeted to be set to 1, when hitting the negative list, level will be 0.
From the 4 examples below, the TimeLogicAdapter two times incorrectly indicates confidence level 1.
Question 1: 'What is 4 + 9?' TimeLogicAdapter returns 1.
Question 2: 'What time is it?' TimeLogicAdapter returns 1.
Question 3: 'What is AI?' TimeLogicAdapter returns 1.
Question 4: 'What is your favorite number?' TimeLogicAdapter returns 0.
While 2 and 4 the Adapter returns the correct confidence level, on question 1 and 3 returning 1 while not hitting the Adapter's default positive list, to me, seems to unexpected behaviour and in case of 3 leading to a incorrect answer. Also, I'm wondering why the Adapter returned 0 in question 4 while it returns 1 in question 3.
Did i miss something or is this unexpected and inconsistent behaviour?
Chatbot Configuration
Logs Training ai.yml: [####################] 100% Training botprofile.yml: [####################] 100% Training computers.yml: [####################] 100% Training conversations.yml: [####################] 100% Training emotion.yml: [####################] 100% Training food.yml: [####################] 100% Training gossip.yml: [####################] 100% Training greetings.yml: [####################] 100% Training health.yml: [####################] 100% Training history.yml: [####################] 100% Training humor.yml: [####################] 100% Training literature.yml: [####################] 100% Training money.yml: [####################] 100% Training movies.yml: [####################] 100% Training politics.yml: [####################] 100% Training psychology.yml: [####################] 100% Training science.yml: [####################] 100% Training sports.yml: [####################] 100% Training trivia.yml: [####################] 100% INFO:chatterbot.chatterbot:MathematicalEvaluation selected "4 + 9 = 13" as a response with a confidence of 1 INFO:chatterbot.chatterbot:TimeLogicAdapter selected "The current time is 11:20 PM" as a response with a confidence of 1 INFO:chatterbot.chatterbot:Beginning search for close text match INFO:chatterbot.chatterbot:Processing search results INFO:chatterbot.chatterbot:Similar text found: What is AI? 0.72 INFO:chatterbot.chatterbot:Using "What is AI?" as a close match to "What is 4 + 9?" with a confidence of 0.72 4 + 9 = 13 INFO:chatterbot.chatterbot:Selecting response from 12 optimal responses. INFO:chatterbot.response_selection:Selecting first response from list of 12 options. INFO:chatterbot.chatterbot:Response selected. Using "Artificial Intelligence is the branch of engineering and science devoted to constructing machines that think." INFO:chatterbot.chatterbot:BestMatch selected "Artificial Intelligence is the branch of engineering and science devoted to constructing machines that think." as a response with a confidence of 0.72 INFO:chatterbot.chatterbot:Not processing the statement using MathematicalEvaluation INFO:chatterbot.chatterbot:TimeLogicAdapter selected "The current time is 11:20 PM" as a response with a confidence of 1 INFO:chatterbot.chatterbot:Beginning search for close text match INFO:chatterbot.chatterbot:Processing search results INFO:chatterbot.chatterbot:Similar text found: Are you sentient? 0.42 INFO:chatterbot.chatterbot:Similar text found: What is your business 0.49 INFO:chatterbot.chatterbot:Similar text found: What is your idea 0.55 INFO:chatterbot.chatterbot:Similar text found: What are you then? 0.59 INFO:chatterbot.chatterbot:Similar text found: What is it like? 0.69 INFO:chatterbot.chatterbot:Using "Are you sentient?" as a close match to "What time is it?" with a confidence of 0.42 The current time is 11:20 PM INFO:chatterbot.chatterbot:Selecting response from 18 optimal responses. INFO:chatterbot.response_selection:Selecting first response from list of 18 options. INFO:chatterbot.chatterbot:Response selected. Using "Sort of." INFO:chatterbot.chatterbot:BestMatch selected "Sort of." as a response with a confidence of 0.42 INFO:chatterbot.chatterbot:Not processing the statement using MathematicalEvaluation INFO:chatterbot.chatterbot:TimeLogicAdapter selected "The current time is 11:20 PM" as a response with a confidence of 1 INFO:chatterbot.chatterbot:Beginning search for close text match INFO:chatterbot.chatterbot:Processing search results INFO:chatterbot.chatterbot:Similar text found: What is AI? 1.0 INFO:chatterbot.chatterbot:Using "What is AI?" as a close match to "What is AI?" with a confidence of 1.0 The current time is 11:20 PM INFO:chatterbot.chatterbot:Selecting response from 12 optimal responses. INFO:chatterbot.response_selection:Selecting first response from list of 12 options. INFO:chatterbot.chatterbot:Response selected. Using "Artificial Intelligence is the branch of engineering and science devoted to constructing machines that think." INFO:chatterbot.chatterbot:BestMatch selected "Artificial Intelligence is the branch of engineering and science devoted to constructing machines that think." as a response with a confidence of 1.0 INFO:chatterbot.chatterbot:Not processing the statement using MathematicalEvaluation INFO:chatterbot.chatterbot:TimeLogicAdapter selected "The current time is 11:20 PM" as a response with a confidence of 0 INFO:chatterbot.chatterbot:Beginning search for close text match INFO:chatterbot.chatterbot:Processing search results INFO:chatterbot.chatterbot:Similar text found: What is your favorite number 0.98 INFO:chatterbot.chatterbot:Using "What is your favorite number" as a close match to "What is your favorite number?" with a confidence of 0.98 I find I'm quite fond of the number 42. INFO:chatterbot.chatterbot:Selecting response from 6 optimal responses. INFO:chatterbot.response_selection:Selecting first response from list of 6 options. INFO:chatterbot.chatterbot:Response selected. Using "I find I'm quite fond of the number 42." INFO:chatterbot.chatterbot:BestMatch selected "I find I'm quite fond of the number 42." as a response with a confidence of 0.98
Process finished with exit code 0