gunthercox / ChatterBot

ChatterBot is a machine learning, conversational dialog engine for creating chat bots
https://chatterbot.readthedocs.io
BSD 3-Clause "New" or "Revised" License
14.1k stars 4.45k forks source link

TimeLogicAdapter: Unexpected return of confidence value 1 #2049

Open AnAnalogGuy opened 4 years ago

AnAnalogGuy commented 4 years ago

Issue: Unexpected return of confidence value 1

My understanding: When creating a TimeLogicAdapter with no additional parameters, the default positive and negative lists are applied. When hitting the positive list, confidence level is expeted to be set to 1, when hitting the negative list, level will be 0.

From the 4 examples below, the TimeLogicAdapter two times incorrectly indicates confidence level 1.

Question 1: 'What is 4 + 9?' TimeLogicAdapter returns 1.

Question 2: 'What time is it?' TimeLogicAdapter returns 1.

Question 3: 'What is AI?' TimeLogicAdapter returns 1.

Question 4: 'What is your favorite number?' TimeLogicAdapter returns 0.

While 2 and 4 the Adapter returns the correct confidence level, on question 1 and 3 returning 1 while not hitting the Adapter's default positive list, to me, seems to unexpected behaviour and in case of 3 leading to a incorrect answer. Also, I'm wondering why the Adapter returned 0 in question 4 while it returns 1 in question 3.

Did i miss something or is this unexpected and inconsistent behaviour?

Chatbot Configuration

import spacy
import logging
from chatterbot import ChatBot
from chatterbot.trainers import ChatterBotCorpusTrainer

logging.basicConfig(level=logging.INFO)
nlp = spacy.load('en_core_web_sm')

english_bot = ChatBot(
    "Luke",
    storage_adapter="chatterbot.storage.SQLStorageAdapter",
    logic_adapters=[
        "chatterbot.logic.MathematicalEvaluation",
        "chatterbot.logic.TimeLogicAdapter",
        "chatterbot.logic.BestMatch"
    ],
    read_only=True)

trainer = ChatterBotCorpusTrainer(english_bot)
trainer.train("chatterbot.corpus.english")

# Print an example of getting one math based response
response = english_bot.get_response('What is 4 + 9?')
print(response)

# Print an example of getting one time based response
response = english_bot.get_response('What time is it?')
print(response)

# Print an example of getting one bestmatch response
response = english_bot.get_response('What is AI?')
print(response)

# Print an example of getting one bestmatch response
response = english_bot.get_response('What is your favorite number?')
print(response)

Logs Training ai.yml: [####################] 100% Training botprofile.yml: [####################] 100% Training computers.yml: [####################] 100% Training conversations.yml: [####################] 100% Training emotion.yml: [####################] 100% Training food.yml: [####################] 100% Training gossip.yml: [####################] 100% Training greetings.yml: [####################] 100% Training health.yml: [####################] 100% Training history.yml: [####################] 100% Training humor.yml: [####################] 100% Training literature.yml: [####################] 100% Training money.yml: [####################] 100% Training movies.yml: [####################] 100% Training politics.yml: [####################] 100% Training psychology.yml: [####################] 100% Training science.yml: [####################] 100% Training sports.yml: [####################] 100% Training trivia.yml: [####################] 100% INFO:chatterbot.chatterbot:MathematicalEvaluation selected "4 + 9 = 13" as a response with a confidence of 1 INFO:chatterbot.chatterbot:TimeLogicAdapter selected "The current time is 11:20 PM" as a response with a confidence of 1 INFO:chatterbot.chatterbot:Beginning search for close text match INFO:chatterbot.chatterbot:Processing search results INFO:chatterbot.chatterbot:Similar text found: What is AI? 0.72 INFO:chatterbot.chatterbot:Using "What is AI?" as a close match to "What is 4 + 9?" with a confidence of 0.72 4 + 9 = 13 INFO:chatterbot.chatterbot:Selecting response from 12 optimal responses. INFO:chatterbot.response_selection:Selecting first response from list of 12 options. INFO:chatterbot.chatterbot:Response selected. Using "Artificial Intelligence is the branch of engineering and science devoted to constructing machines that think." INFO:chatterbot.chatterbot:BestMatch selected "Artificial Intelligence is the branch of engineering and science devoted to constructing machines that think." as a response with a confidence of 0.72 INFO:chatterbot.chatterbot:Not processing the statement using MathematicalEvaluation INFO:chatterbot.chatterbot:TimeLogicAdapter selected "The current time is 11:20 PM" as a response with a confidence of 1 INFO:chatterbot.chatterbot:Beginning search for close text match INFO:chatterbot.chatterbot:Processing search results INFO:chatterbot.chatterbot:Similar text found: Are you sentient? 0.42 INFO:chatterbot.chatterbot:Similar text found: What is your business 0.49 INFO:chatterbot.chatterbot:Similar text found: What is your idea 0.55 INFO:chatterbot.chatterbot:Similar text found: What are you then? 0.59 INFO:chatterbot.chatterbot:Similar text found: What is it like? 0.69 INFO:chatterbot.chatterbot:Using "Are you sentient?" as a close match to "What time is it?" with a confidence of 0.42 The current time is 11:20 PM INFO:chatterbot.chatterbot:Selecting response from 18 optimal responses. INFO:chatterbot.response_selection:Selecting first response from list of 18 options. INFO:chatterbot.chatterbot:Response selected. Using "Sort of." INFO:chatterbot.chatterbot:BestMatch selected "Sort of." as a response with a confidence of 0.42 INFO:chatterbot.chatterbot:Not processing the statement using MathematicalEvaluation INFO:chatterbot.chatterbot:TimeLogicAdapter selected "The current time is 11:20 PM" as a response with a confidence of 1 INFO:chatterbot.chatterbot:Beginning search for close text match INFO:chatterbot.chatterbot:Processing search results INFO:chatterbot.chatterbot:Similar text found: What is AI? 1.0 INFO:chatterbot.chatterbot:Using "What is AI?" as a close match to "What is AI?" with a confidence of 1.0 The current time is 11:20 PM INFO:chatterbot.chatterbot:Selecting response from 12 optimal responses. INFO:chatterbot.response_selection:Selecting first response from list of 12 options. INFO:chatterbot.chatterbot:Response selected. Using "Artificial Intelligence is the branch of engineering and science devoted to constructing machines that think." INFO:chatterbot.chatterbot:BestMatch selected "Artificial Intelligence is the branch of engineering and science devoted to constructing machines that think." as a response with a confidence of 1.0 INFO:chatterbot.chatterbot:Not processing the statement using MathematicalEvaluation INFO:chatterbot.chatterbot:TimeLogicAdapter selected "The current time is 11:20 PM" as a response with a confidence of 0 INFO:chatterbot.chatterbot:Beginning search for close text match INFO:chatterbot.chatterbot:Processing search results INFO:chatterbot.chatterbot:Similar text found: What is your favorite number 0.98 INFO:chatterbot.chatterbot:Using "What is your favorite number" as a close match to "What is your favorite number?" with a confidence of 0.98 I find I'm quite fond of the number 42. INFO:chatterbot.chatterbot:Selecting response from 6 optimal responses. INFO:chatterbot.response_selection:Selecting first response from list of 6 options. INFO:chatterbot.chatterbot:Response selected. Using "I find I'm quite fond of the number 42." INFO:chatterbot.chatterbot:BestMatch selected "I find I'm quite fond of the number 42." as a response with a confidence of 0.98

Process finished with exit code 0

alienhunter3010 commented 4 years ago

Similar behaviour. Simple question "Who are you?" as confidence: 1 from TimeLogicAdapter

ChristianTremblay commented 3 years ago

Same here Duplicate issue #2038 Possible solution issue #1830