davisking
commented
3 years ago It's not going to just memorize the training data. MITIE is doing many rounds of cross-validation to find a model that can be expected to generalize well. It's not trying to fit the training dataset as hard as it can, as that will overfit and work badly on test datasets. You need to give more training data.
On Sun, Dec 13, 2020 at 9:16 PM AdamMiltonBarker notifications@github.com wrote:
Expected Behavior
Trained model should classify all trained entities Current Behavior
Trained model does not classify all trained entities Steps to Reproduce
Below is the training input:
` Turn on the lamp in the bedroom {'entity': 'State', 'rangeFrom': 1, 'rangeTo': 2} {'entity': 'Device', 'rangeFrom': 3, 'rangeTo': 4} {'entity': 'Zone', 'rangeFrom': 6, 'rangeTo': 7}
Turn the lamp in the bedroom off {'entity': 'Device', 'rangeFrom': 2, 'rangeTo': 3} {'entity': 'Zone', 'rangeFrom': 5, 'rangeTo': 6} {'entity': 'State', 'rangeFrom': 6, 'rangeTo': 7}
Can you turn the lamp in the bedroom on {'entity': 'Device', 'rangeFrom': 4, 'rangeTo': 5} {'entity': 'Zone', 'rangeFrom': 7, 'rangeTo': 8} {'entity': 'State', 'rangeFrom': 8, 'rangeTo': 9}
Will you turn the lamp in the bedroom off {'entity': 'Device', 'rangeFrom': 4, 'rangeTo': 5} {'entity': 'Zone', 'rangeFrom': 7, 'rangeTo': 8} {'entity': 'State', 'rangeFrom': 8, 'rangeTo': 9}
Please will you turn the lamp in the bedroom on {'entity': 'Device', 'rangeFrom': 5, 'rangeTo': 6} {'entity': 'Zone', 'rangeFrom': 8, 'rangeTo': 9} {'entity': 'State', 'rangeFrom': 9, 'rangeTo': 10}
Will you turn off the lamp in the bedroom {'entity': 'State', 'rangeFrom': 3, 'rangeTo': 4} {'entity': 'Device', 'rangeFrom': 5, 'rangeTo': 6} {'entity': 'Zone', 'rangeFrom': 8, 'rangeTo': 9}
Please turn on the lamp in the bedroom {'entity': 'State', 'rangeFrom': 2, 'rangeTo': 3} {'entity': 'Device', 'rangeFrom': 4, 'rangeTo': 5} {'entity': 'Zone', 'rangeFrom': 7, 'rangeTo': 8}
Please turn the lamp in the bedroom off {'entity': 'Device', 'rangeFrom': 3, 'rangeTo': 4} {'entity': 'Zone', 'rangeFrom': 6, 'rangeTo': 7} {'entity': 'State', 'rangeFrom': 7, 'rangeTo': 8}
What is the temperature in the bedroom? {'entity': 'Measurement', 'rangeFrom': 3, 'rangeTo': 4} {'entity': 'Zone', 'rangeFrom': 6, 'rangeTo': 7}
What is the temperature in the office? {'entity': 'Measurement', 'rangeFrom': 3, 'rangeTo': 4} {'entity': 'Zone', 'rangeFrom': 6, 'rangeTo': 7}
Tell me the humidity in the bedroom? {'entity': 'Measurement', 'rangeFrom': 3, 'rangeTo': 4} {'entity': 'Zone', 'rangeFrom': 6, 'rangeTo': 7}
Tell me the humidity in the office? {'entity': 'Measurement', 'rangeFrom': 3, 'rangeTo': 4} {'entity': 'Zone', 'rangeFrom': 6, 'rangeTo': 7}
Can you please tell me temperature in the bedroom is? {'entity': 'Measurement', 'rangeFrom': 5, 'rangeTo': 6} {'entity': 'Zone', 'rangeFrom': 8, 'rangeTo': 9}
Can you please tell me temperature in the office is? {'entity': 'Measurement', 'rangeFrom': 5, 'rangeTo': 6} {'entity': 'Zone', 'rangeFrom': 8, 'rangeTo': 9}
Will you tell me temperature in the bedroom is? {'entity': 'Measurement', 'rangeFrom': 4, 'rangeTo': 5} {'entity': 'Zone', 'rangeFrom': 7, 'rangeTo': 8}
Will you tell me humidity in the bedroom is? {'entity': 'Measurement', 'rangeFrom': 4, 'rangeTo': 5} {'entity': 'Zone', 'rangeFrom': 7, 'rangeTo': 8} `
Classification input/output:
` {'Response': 'OK', 'ResponseData': [{'Received': 'Turn on the lamp in the bedroom', 'Intent': 'TurnOnIn', 'Confidence': '1.0', 'Response': 'One minute also', 'Context': [{'In': '', 'Out': '', 'Current': ''}], 'Extension': None, 'Entities': [{'Entity': 'State', 'ParsedEntity': 'on', 'Confidence': '0.089'}, {'Entity': 'Device', 'ParsedEntity': 'lamp', 'Confidence': '0.101'}]}]}
{'Response': 'OK', 'ResponseData': [{'Received': 'Turn the lamp in the bedroom off', 'Intent': 'TurnOnIn', 'Confidence': '0.9999999', 'Response': 'One minute also', 'Context': [{'In': '', 'Out': '', 'Current': ''}], 'Extension': None, 'Entities': [{'Entity': 'Device', 'ParsedEntity': 'lamp', 'Confidence': '0.115'}, {'Entity': 'Zone', 'ParsedEntity': 'bedroom', 'Confidence': '0.084'}]}]}
{'Response': 'OK', 'ResponseData': [{'Received': 'Can you turn the lamp in the bedroom on', 'Intent': 'TurnOnIn', 'Confidence': '0.9999999', 'Response': 'One minute also', 'Context': [{'In': '', 'Out': '', 'Current': ''}], 'Extension': None, 'Entities': [{'Entity': 'Device', 'ParsedEntity': 'lamp', 'Confidence': '0.115'}, {'Entity': 'Zone', 'ParsedEntity': 'bedroom', 'Confidence': '0.083'}]}]}
{'Response': 'OK', 'ResponseData': [{'Received': 'Will you turn the lamp in the bedroom off', 'Intent': 'TurnOnIn', 'Confidence': '0.9999999', 'Response': 'One minute also', 'Context': [{'In': '', 'Out': '', 'Current': ''}], 'Extension': None, 'Entities': [{'Entity': 'Device', 'ParsedEntity': 'lamp', 'Confidence': '0.115'}, {'Entity': 'Zone', 'ParsedEntity': 'bedroom', 'Confidence': '0.084'}]}]} `
Etc, it is rarely detecting all trained entities.
-
Version: 0.7.0
Where did you get MITIE: Github
Platform: No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 18.04.5 LTS Release: 18.04 Codename: bionic
Compiler: gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/mit-nlp/MITIE/issues/213, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABPYFR4SBJAPPKVAZBINKJTSUVYOVANCNFSM4UZ77WTQ .
AdamMiltonBarker
Expected Behavior
Trained model should classify all trained entities
Current Behavior
Trained model does not classify all trained entities
Steps to Reproduce
I use MITIE in my custom intent classifier. Below is an example training input for MITIE:
Classification input/output:
Etc, it is rarely detecting all trained entities.
Version: 0.7.0
Where did you get MITIE: Github
Platform: No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 18.04.5 LTS Release: 18.04 Codename: bionic
Compiler: gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0