talmo
commented
4 years ago Hi @z2357,
I'm glad that you're liking SLEAP!
question 1: there is always a data overflow problem when I applied it on my 2688 x 1520, RGB, 30min, video data with 10 manual labels. I set the batch size to be 1 and stride 4, which is really low but it does not work.
Yes, that's definitely pretty big for a video.
What GPU are you using?
One thing to try is to apply input scaling (e.g., 0.5x) to downsample the images before feeding them into the neural network.
The other solution is to use the top-down approach (if you're currently using bottom-up). In top-down, you can train a network to predict the centroid/anchor point from a relatively low resolution image (e.g., 0.25x input scale) and then use slightly higher resolution crops for the instance-centered confidence maps.
question 2: I cut out a 2-minute video slice and completed training but the training accuracy plot disappeared after the program finished. How to get it back?
The training accuracy history is also saved in a CSV file in the same folder as the model, and you can enable TensorBoard logging as well.
question 3: I run inference after the training, but there is no predicted label showing up. There is a list of instances and colorful dots on the predicted frames. But I am not able to draw the predictions on the video or anywhere.
For this we might need to do some more troubleshooting but the first thing to check is the command prompt to see what messages appeared.
While it does seem to be detecting instances, points are not being detected which is odd.
If you're using bottom-up this shouldn't happen, but if you are then paste a screenshot of the skeleton panel in the GUI and we'll go from there.
If you're using top-down it's possible this may happen if the instance-centered network is having issues after training.
Let me know which one you're using and if it's top-down, maybe share the configuration (e.g., training_job.json in the model folder).
question 4: there is a developer installation. Is there a tutorial on how to run the scripts directly and in particular how to save the predictions on the videos?
Yes! We have tutorials on programmatic training and inference on Colab or on your own computer/cluster. You may also find this workshop tutorial helpful since it has a few more examples of interactive inference (e.g, saving predictions on the video).
Cheers,
Talmo
Thanks a lot for the developer of this framework. It is a great achievement in animal pose estimation. I used the GUI application on my dataset. question 1: there is always a data overflow problem when I applied it on my 2688 x 1520, RGB, 30min, video data with 10 manual labels. I set the batch size to be 1 and stride 4, which is really low but it does not work. question 2: I cut out a 2-minute video slice and completed training but the training accuracy plot disappeared after the program finished. How to get it back? question 3: I run inference after the training, but there is no predicted label showing up. There is a list of instances and colorful dots on the predicted frames. But I am not able to draw the predictions on the video or anywhere. question 4: there is a developer installation. Is there a tutorial on how to run the scripts directly and in particular how to save the predictions on the videos?