b-quachtran commented 3 years ago

Resolves Rule / Story conflicts in Helpdesk Assistant
Reverts Rasa version from 2.2.0 to 2.1.3 (this issue is causing the pipeline to fail: https://github.com/RasaHQ/rasa/issues/7603, so reverting for now)

github-actions[bot] commented 3 years ago

Commit: 0cc3bf91d49fe6972493308915479d000a78b7ac Data: default

Configuration	Intent Classification Micro F1	Entity Recognition Micro F1	Response Selection Micro F1	Story Recognition Micro F1
`config.yml`	0.8647 (0.00)	0.8182 (0.00)	`no data`	1.0000 (0.00)

Intent Cross-Validation Results

class	support	f1-score	confused_with
macro avg	133	0.8533	N/A
weighted avg	133	0.8611	N/A
open_incident	15	0.8750	problem_email(1)
inform	13	0.8462	incident_status(1), deny(1)
incident_status	13	0.8800	open_incident(2)
out_of_scope	12	0.9565	affirm(1)
affirm	12	0.7826	thank(2), deny(1)
problem_email	10	0.9524	N/A
password_reset	9	1.0000	N/A
greet	9	0.8235	thank(1), help(1)
deny	8	0.8235	inform(1)
help	7	0.9333	N/A
human_handoff	7	0.6000	bot_challenge(2), open_incident(1)
goodbye	6	1.0000	N/A
bot_challenge	6	0.8571	N/A
thank	6	0.6154	inform(1), greet(1)

Entity Cross-Validation Results

entity	support	f1-score	precision	recall
priority	13	0.8182	1	0.6923
micro avg	13	0.8182	1	0.6923
macro avg	13	0.8182	1	0.6923
weighted avg	13	0.8182	1	0.6923

github-actions[bot] commented 3 years ago

Commit: 07f97185804903dd3b0a3f07e9420f03256b6b14 Data: default

Configuration	Intent Classification Micro F1	Entity Recognition Micro F1	Response Selection Micro F1	Story Recognition Micro F1
`config.yml`	0.8346 (0.00)	0.7000 (0.00)	`no data`	1.0000 (0.00)

class	support	f1-score	confused_with
macro avg	133	0.8386	N/A
weighted avg	133	0.8353	N/A
open_incident	15	0.8667	problem_email(1), incident_status(1)
inform	13	0.7500	thank(2), open_incident(1)
incident_status	13	0.8148	open_incident(1), inform(1)
out_of_scope	12	0.9167	incident_status(1)
affirm	12	0.6667	thank(1), incident_status(1)
problem_email	10	0.9524	N/A
password_reset	9	1.0000	N/A
greet	9	0.8889	thank(1)
deny	8	0.5714	affirm(2), thank(1)
human_handoff	7	0.9231	affirm(1)
help	7	0.8571	affirm(1)
thank	6	0.5333	out_of_scope(1), help(1)
bot_challenge	6	1.0000	N/A
goodbye	6	1.0000	N/A

entity	support	f1-score	precision	recall
priority	13	0.7000	1	0.5385
micro avg	13	0.7000	1	0.5385
macro avg	13	0.7000	1	0.5385
weighted avg	13	0.7000	1	0.5385

indam23 commented 3 years ago

My bad on bumping the rasa version