How to pause and step through scenario generation to inspect generated images *with labels* showing?

[lessw2020]

When I run a scenario, I can see the labels for the bounding boxes flashing past as each image is being created (since I have display visualizations checked). However, it goes so fast I can't get any useful info from that....just a blur of images and yellow boxes. I thus would like to step through so that I can see each individual image, so that I can actually see the labels to inspect that things are happening as expected.

However, if I pause, then hit play, and thus step through each image, I can see the individual images, but no bounding box labels are shown on the generated image. Is there a way to make that happen - i.e. pause and step such that the bounding box labels are displayed on each new generated image for review?

[StevenBorkman]

Hi Less, thanks for reaching out. Unfortunately right now, we really don't have a way to step through frames in the Unity Editor with the visualization info turned on. We have a ticket opened for this in our backlog and hope to implement it sometime in the future.

Until then, you can visualize your frame by frame results using our dataset insights tool. If you are not familiar with it, I would suggest looking at Step 8 of Phase I in our Perception Tutorial. It walks you through the process of bringing your data into dataset insights and the different analyzers that we have in the perception jupyter notebook. The included image is of the bounding box visualizer.

Sorry, there isn't a more convenient way to do this in the Unity Editor at the current time, but it is on our roadmap. Please feel free to reach out to us if you have anymore questions.

Steve

[lessw2020]

Hi @StevenBorkman, Thanks much for the info that this is pending fix in a future release. I found that I could somewhat swap out frame inspection, by manually reviewing each model by hand with rotation in a scene...bit tedious but works and then I use our own show_batch code to review pre-training for the obj detection.
Direct frame inspection would still be ideal though. I will check out the insight visualization tool. I did use it for stats as a quick test, but hadn't looked at the visualization. One related item - the docker image for this tooling is huge (5GB)? Any chance the insights notebook will just be released as a notebook with a requirements.txt for any supporting files, to avoid all that massive docker overhead?
Thanks again for the update about it being on the roadmap, much appreciated and look forward to the future release!

[adason]

@lessw2020

One related item - the docker image for this tooling is huge (5GB)? Any chance the insights notebook will just be released as a notebook with a requirements.txt for any supporting files, to avoid all that massive docker overhead?

The datasetinsights repo include notebooks and instructions on how to run notebooks locally. You can find them under /notebooks directory. You will need to create a virtual environment by following instructions under CONTRIBUTING.md. There are one caveat: the windows virtual environment was not fully tested due to issue https://github.com/Unity-Technologies/datasetinsights/issues/48. It should work for mac or linux.

The docker image is huge due to heavy model training dependencies. The docker base image including CUDA is roughly 1GB. Pytorch and Tensorflow add 1GB each. That’s 3GB already without other dependencies. We are working on removing model training dependencies in datasetinsights which should greatly reduce the docker image size.

[JonathanHUnity]

Support for the "Step" button in the Unity Editor is now present on the master branch and will be in our next release. Cheers!

[lessw2020]

@JonathanHUnity - that is a huge help. Thanks for implementing this!