issue with Test categorizers

[fengaccoumt]

Good afternoon! Thanks for a wonderful piece of programming.

I get trained categorizer now. But when I want use the ground up example to test trained categorizer, I can't get the result. In terminal it like this:

This is my trained categorizer report in terminal :

This is my ground up example‘s name:

I want to know how to solution this question?

Thanks you a lot!

[yujiahu415]

Hi,

1. You need to have at least two categories to enable the training / testing. It seems you only used one behavior category for testing.
2. You need to make the behavior name in testing dataset matches that in the Categorizer. The behavior name in your testing dataset is 'limptail_hindleg' while in your terminal it says 'behavior name in the Categorizer is hindleg'.

[fengaccoumt]

Thanks for your answers.

But the problem also remain. This time I use four categories and harmonize the behavior name of Categorizer and testing dataset , but it also show unused behavior names in the Categorizer in terminal like this.

This is my behavior name for training Categorizer:

This is my ground up example's behavior name:

Thanks you a lot!

[yujiahu415]

Behavior names should be English letters (can contain "-" or " "). But please do not use numbers, special characters such as "#", or "_". Let me know if you still have the issue when you use English letters.

[fengaccoumt]

Thanks for your answers.

Sorry, I still have the issue when I use English letters, the error like this in terminal.

Thanks you a lot!

[fengaccoumt]

The hole error in terminal is like this.

[yujiahu415]

The error message indicated that one image file (you can find the name in the error message) in the "aa" folder was either broken, or has a path that could not be read properly. Please check that file, either replace or remove that file together with its associated "animation" file.

[fengaccoumt]

Thanks for your answers.

But the new error come out. It like this in terminal.

Thanks you a lot!

[fengaccoumt]

Thanks for your help, I get training report now. But the value of precision, recall and f1-score is very low, how can I get the accuracy good Categorizer?

Thanks you a lot!

[yujiahu415]

Increase the number of training examples and sort them well. Typically around 100 pairs for each behavior category. You may also increase the number of your testing examples, that will give the test more statistical power. Currently there's only three examples for each category in your testing data.

[fengaccoumt]

Thank you very much for your help, now I have been able to get the test report, and increase the accuracy to 0.91.

[yujiahu415]

Thanks for letting me know this. Great to hear that!

[fengaccoumt]

Hello!

I would like to ask if using different frames of behavior examples to train categorizers will affect the performance of the model?

Thanks for your reply.

Yujia Hu @.***> 于2024年2月7日周三 10:08写道：

Thanks for letting me know this. Great to hear that!

— Reply to this email directly, view it on GitHub https://github.com/umyelab/LabGym/issues/89#issuecomment-1931062479, or unsubscribe https://github.com/notifications/unsubscribe-auth/BFS3XYKACJWE2O64ECBI5PLYSLHYNAVCNFSM6AAAAABCSJRRFSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZRGA3DENBXHE . You are receiving this because you authored the thread.Message ID: @.***>

[yujiahu415]

Hello! I would like to ask if using different frames of behavior examples to train categorizers will affect the performance of the model? Thanks for your reply. Yujia Hu @.> 于2024年2月7日周三 10:08写道： … Thanks for letting me know this. Great to hear that! — Reply to this email directly, view it on GitHub <#89 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/BFS3XYKACJWE2O64ECBI5PLYSLHYNAVCNFSM6AAAAABCSJRRFSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZRGA3DENBXHE . You are receiving this because you authored the thread.Message ID: @.>

What do you mean by "using different frames of behavior example"? Did you mean using behavior examples of different fps? Or just different behavior examples? How the trained Categorizer performs depends on how you select and sort the behavior examples.

[fengaccoumt]

Hello!

I want to evaluate the categoriser's performance, in our programe we just can use 80% samples to training the categoriser, and 20% to generate the test report. But my prefessor recommend me to use 70% samples to training and 30% to generate the test report. I don't know if this number will have some effect on evaluate the categoriser's performance. I want ask for your advise about this.

Thanks for your reply!

Yujia Hu @.***> 于2024年7月4日周四 10:15写道：

Hello! I would like to ask if using different frames of behavior examples to train categorizers will affect the performance of the model? Thanks for your reply. Yujia Hu @.

> 于2024年2月7日周三 10:08写道： … <#m-807210377779854194> Thanks for letting me know this. Great to hear that! — Reply to this email directly, view it on GitHub <#89 (comment) https://github.com/umyelab/LabGym/issues/89#issuecomment-1931062479>, or unsubscribe https://github.com/notifications/unsubscribe-auth/BFS3XYKACJWE2O64ECBI5PLYSLHYNAVCNFSM6AAAAABCSJRRFSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZRGA3DENBXHE https://github.com/notifications/unsubscribe-auth/BFS3XYKACJWE2O64ECBI5PLYSLHYNAVCNFSM6AAAAABCSJRRFSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZRGA3DENBXHE . You are receiving this because you authored the thread.Message ID: @.>

What do you mean by "using different frames of behavior example"? Did you mean using behavior examples of different fps? Or just different behavior examples? How the trained Categorizer performs depends on how you select and sort the behavior examples.

— Reply to this email directly, view it on GitHub https://github.com/umyelab/LabGym/issues/89#issuecomment-2207778155, or unsubscribe https://github.com/notifications/unsubscribe-auth/BFS3XYJQVGXPET6LTEF5WG3ZKSO2XAVCNFSM6AAAAABKIRJDSOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMBXG43TQMJVGU . You are receiving this because you authored the thread.Message ID: @.***>

[yujiahu415]

The more examples for testing, the more statistical power of the testing. In the other hand, the more examples for training, the better performance of the training. Generally speaking, 100 pairs of examples for each behavior category for testing should give you sufficient statistical power, and all the resting examples can be used for training.