Pose didn't move or Average Intensity Too Extreme

[arjunds]

I've ran SceneNetRGBD about 10 times now and keep getting either one of those 2 errors. I know it happens because it's completely random, but is there anyway to make it more constrained and force it not to be a super bright scene, or until it creates a valid camera trajectory, it keeps running. I could keep running it until I get a valid scene and trajectory, but I want to be able to create multiple scenes/trajectories and I already wait a long time for the 'Simulating System' part in scene_generator, only for it to fail either when creating the camera trajectory or rendering. Any help would be appreciated. Thanks in advance!

[juanhotencoding]

They do state in the README: "If the rendering does not output anything it may say "Average intensity too extreme". This is because of the checks put in to ensure the images are not too bleached or dark. Just keep rerunning the command and a new random seed will generate a different set of lights."

I'm right now trying to build a script that takes those errors into account and just re-running the commands if the exit status is not 0 or if one of the text files wasn't created (i.e., scene_description.txt and scene_and_camera_trajectory.txt)

[arjunds]

Yeah I noticed that in the README, but that doesn't explain why everytime I run it, I get some error. It also happened when I used their libcvd library so I'm going to try and revert back to the original one and hopefully that allows me to get some rendered images.

[jmccormac]

I can think of a few things that may help here. First of all, a direct way is to change the thresholds on either the brightness requirements (line 82-84 of scenenet_render.cpp) or the valid trajectory requirements (line 566-627 of camera_trajectory_generator.cpp) so that they are a bit looser, depending on the kind of scenes you want. A more advanced variant which would require some additional coding would be to pass back information about the lights being too dim or bright and then change the scalar in that direction, or backtrack the trajectory simulation and start again, but this would require a bit of hacking.

I think a different approach to scene generation is the best option though. When generating a very many scenes I did not find either of these two errors particularly onerous because it is not necessary for all stages to complete in a single pass. Instead the method I used is to first run the scene_generator in a loop (a shell script works fine) to generate thousands of scene_description_[id].txt files - to do this you modify the given main function scenenetrgb-d/scene_generator/scenenet_room_generator.cpp to accept a third input argument denoting the id and change the "filename_path" variable to have the appended suffix in the name.

You can then have another bash script (or some form of 'worker') which takes the completed scene_description files and tries to generate a camera trajectory for them (again with an appropriate suffix and slight modification to scenenetrgb-d/camera_trajectory_generator/camera_trajectory_generator.cpp) - if the trajectory succeeds you can remove the scenedescription[id].txt file (or if you want multiple trajectories for a single scene you can leave the file in place and sample with replacement), and if it fails you leave it there to try to create a different random trajectory in the same scene again. The same approach will work when generating the random lights and performing the intensity check for rendering but reading the output scene_and_trajectory_description_[id]_[trajectory_id].txt file.

Hope this helps!

[arjunds]

Alright, I think the bash scripts will be the most effective, though I will also try to modify the requirements, as specified above. Also, for the times where the Pose/Target move out of bounds, couldn't you simulate a 'bounce' (as you do when hitting an object according to the code) so that the camera stays within the scene and there's no need to rerun? I haven't looked too closely at camera_trajectory_generator.cpp but is that something that is possible?

[jmccormac]

It is possible to simulate a bounce - the complication is defining a suitable criteria. The scenes are not convex and so it is a bit complicated to automatically calculate the bounding volume - although given there are not many scenes you could also manually specify a set of finite planes for each scene and check the intersection against each.

Alternatively you could use another heuristic. Maybe an effective approach could be to cause a collision if there ever is no valid depth along the world y-axis (i.e. if there is ever not a floor or roof, it has left the confines of the scene and reversing will hopefully keep you in it).

[arjunds]

Yes, sorry, collision was what I meant when I said bounce. I noticed that you just reversed the velocity when hitting an object and assumed you were trying to model a bounce in a simple way. Theoretically, that should keep the camera in bounds like you said, so I'll give that a shot. Thanks for the help!