I've spent a significant amount of time trying to improve the performance of my classifier, but the results are still unsatisfactory. I'm reaching out to request your assistance in identifying where the issues might be.
Project Details:
Behavior Classes: I'm attempting to label 8 specific behaviors, plus an "other" category, totaling 9 classes.
Although the results were acceptable in terms of performance (%), when the data were actually examined from the predict module, the results were not very good. While the classifier performed relatively well on the training set (documents that were fully annotated by hand), I noticed a significant drop in the proportion of ‘other’ class. Specifically, the model predicted 14.2% of the ‘other’ class, whereas the actual proportion is 37%. Performance deteriorates further on data outside of the training set. The ‘other’ class is predicted to be less than 1 %.
Attempted Solutions: I've utilized the refine function and added 20 manually corrected videos to improve the model, but the performance has not improved as expected.
Request:
I'm sincerely seeking your help to understand and resolve these issues. Any guidance on how to enhance the classifier's performance, especially regarding the confusion between "other" class and "tonic immobility" class, would be greatly appreciated.
Thank you for your time and assistance!
Best regards,
Desktop:
OS: [WSL2]
Browser [Microsoft Edge]
Version [latest]
Project Config (please post the content of the corresponding config.ini file)
Hello,
I've spent a significant amount of time trying to improve the performance of my classifier, but the results are still unsatisfactory. I'm reaching out to request your assistance in identifying where the issues might be.
Project Details:
Behavior Classes: I'm attempting to label 8 specific behaviors, plus an "other" category, totaling 9 classes.
Annotated Sample Distribution:
Being attacked: 2,984.0 Being investigated: 7,610.4 Defensive attack: 2,188.8 Flee: 2,549.6 Freezing: 6,486.4 Rotating: 432.0 Tonic immobility: 13,706.4 Grooming: 792.8 Other: 21,403.2
Issues Encountered:
Although the results were acceptable in terms of performance (%), when the data were actually examined from the predict module, the results were not very good. While the classifier performed relatively well on the training set (documents that were fully annotated by hand), I noticed a significant drop in the proportion of ‘other’ class. Specifically, the model predicted 14.2% of the ‘other’ class, whereas the actual proportion is 37%. Performance deteriorates further on data outside of the training set. The ‘other’ class is predicted to be less than 1 %.
Attempted Solutions: I've utilized the refine function and added 20 manually corrected videos to improve the model, but the performance has not improved as expected.
Request:
I'm sincerely seeking your help to understand and resolve these issues. Any guidance on how to enhance the classifier's performance, especially regarding the confusion between "other" class and "tonic immobility" class, would be greatly appreciated.
Thank you for your time and assistance!
Best regards,
Desktop:
Project Config (please post the content of the corresponding config.ini file)