Question with "Generate Behavior Examples"

[dylankimnguyen]

When Generating Behavior examples, it asks me to select a video/images to generate behavior examples, what video/images would this be? The original video? Or the preprocessed video? A new video with the same animals? Or the generated image examples?

Another question, sometimes when I am generating behavior examples, it seems slow, and the terminal looks like this, and the GUI freezes, does this indicate that the behavior examples are still generating and I should wait? Will there be a message saying that the process is complete?

[yujiahu415]

If you want to analyze behaviors in static images, you use images to generate behavior examples; if you want to analyze behaviors in videos, you use videos to generate behavior examples. You can use exactly the same settings for both generating behavior examples and analyzing behaviors. That is, if you want to use preprocessed videos during analysis, you use preprocessed videos to generate behavior examples and do the training. If you don’t want to use preprocessed videos during analysis, then just don’t do preprocessing.

If the gui freezes, it means the program is in process. When it finishes, there will be a message of completion in the CMD, and the gui will be unfrozen. If you’re using Detectors, you need a good GPU to achieve decent speed.

[dylankimnguyen]

Thank you for the response, also. if I do not have a decent GPU would you just recommend me to use the subtract background method? I tried this before and when categorizing the unsorted behavior and sorting it, I am having some problems. Am I trying to sort the images in "examples" or the images within the behavior folder? These were created through "Generate behavior examples"

After I have generated behavior examples, I realized that from the video I am using that the output is detecting change in the background, rather than the animal. In the image below, it is only the tree, rather than the orangutan which on the right which seems to be cut out. Do you have any idea how I can fix this? Would I have to edit and crop the original video to only the orangutan?

[yujiahu415]

Background subtraction based detection method can only be used in videos with static background and static illumination. So if the background or illumination varies in the video, you need to choose Detector based detection method. If you don’t have a good GPU, you can downsize the video frame size and use smaller inferencing frame size (like 320) when training a Detector. This can guarantee speed at the cost of lower detection accuracy. And for your question, I don’t quite get it. Can you let me know the settings for ‘generating behavior examples’? And the image you showed, is it an extracted background image output from LabGym?

[dylankimnguyen]

Thanks for the reply, I think that I will be using the Detector based detection method as the background is not fully static and may illuminate, as before I used the subtract background method, which seems to be the error, also causing the wrong results.

[dylankimnguyen]

Another question, regarding "Sort Behavior Examples", it requires me to select a folder from the "Generate Behavior Examples", however, in the generate behavior examples part, I used the Detector method, thus the folders I got from the output already have the behavior sorted (Unlike when using the background subtraction method where I only got one unsorted behavior examples folder). How to proceed from here?

[dylankimnguyen]

I am trying to train the categorizer and it stopped after 5 iterations, though my computer is not great so I will be using my university PC in the near future, but these are the current errors. It seems that it is not detecting any behavior names, however, there should be 3 behavior names?

This is the settings:

This is some of the folder from "prepare training examples":

Folder which contains sorted behavior examples:

[yujiahu415]

Hi, You can try to delete the "_0" in the behavior names, otherwise LabGym reads "0" as the behavior name, as shown in the terminal outputs: "Found behavior name ["0"]". By the way, why there's a "_0" in your sorted behavior folders? Did you train a Detector to detect these three behaviors? In fact, Detector is used to detect the animals / objects in your videos, like mouse / monkey, not the behaviors.

[dylankimnguyen]

I did use the detector to detect these three behaviours. Should I have not done this and instead used it only to detect the orangutan? And should i delete all the "_0" everywhere? Or only in "prepared training examples" folder?

[yujiahu415]

Yes you can extract frames and use Roboflow to annotate "orangutan" in the images and use the annotated images to train a Detector. After that, select the trained Detector to generate behavior examples and sort them into different categories. After that, use the sorted behavior examples to train a Categorizer and you don't need to worry about the "_0" because the behavior examples will not have that if you do this way.

[dylankimnguyen]

Okay thank you, I will try it this way! Also when annotating images using Robloflow for the "orangutan", should I use a wide variety of frames where the orangutan is doing something different, to ensure the Detector can detect the orangutan in many different frames?

[yujiahu415]

Yes, you need to select frames that cover different scenarios, i.e., the orangutans are doing different things and are in different background. This can help the trained Detector to detect orangutans in different scenarios.

[dylankimnguyen]

Thanks, I will let you know if I have any other questions in the future

[yujiahu415]

Sure! No problem!