Vision Sensor causes slow steps, and different speeds in general

[bycn]

When I create a Vision sensor in the environment, even if I don't invoke capture_rgb(), the task steps become dramatically slower. Using cProfile shows some of the calls taking .17s and the others taking 2-4 seconds, and this is occurring in the task.step() function (and further down the stack, in sim.py extstep.) In comparison, without, it takes about ~0.008 for some and ~0.155 for others. Could you explain the discrepancy in the speeds for the vision, as well as the discrepancy in the speeds between the calls?

I tried to work around this, and in the scenario that I load in the Vision sensor in the task (in the ttm) itself, I see the same calls to the step (not the capture_rgb()) taking .02s & .6s. Is there rendering being done in the step? This is still too slow for our purposes. What is the proper way to add a vision sensor to speed up as fast as possible?

Debugging: Looks like the calls that take longer are due to calling extstep 20 times versus once. I'm also seeing:

 pyrep.py:209: UserWarning: Could no
t change simulation timestep. You may need to change it to "custom dt" using simulation
 settings dialog. 

which may be related.

[stepjam]

I just pushed something that will help with:

When I create a Vision sensor in the environment, even if I don't invoke capture_rgb(), the task steps become dramatically slower.

This should no longer be the case.

To get the fastest step time possible

• Only enable the cameras that you need using the ObservationConfig.
• Using the OPENGL renderer over the OPENGL3 rendrer will be much faster. The former uses shadows which does increase rendering time.
• If you must have shadows, then decreasing the number of lights will decrease rendering time.

[bycn]

Thanks. Any ideas for the different timings for steps?

[stepjam]

Not sure what you mean; but on my desktop I get 0.07s per step with all observations enabled (rgb, depth, mask) for all 4 cameras (OPENGL3 renderer), using the joint velocity action mode.

[bycn]

pyrep.py:209: UserWarning: Could no t change simulation timestep. You may need to change it to "custom dt" using simulation settings dialog.

Maybe related to this? Basically it’s calling simextstep ~20 times on some of the steps..

[stepjam]

Hi, It's unrelated to that. When you send a open/close command, this steps the simulator several times. So could be that.

Sent from my phone (please excuse any typos!)

---

From: Bryan Chen notifications@github.com Sent: Wednesday, April 1, 2020 9:43:00 PM To: stepjam/RLBench RLBench@noreply.github.com Cc: Stephen James sjames_93@hotmail.co.uk; Comment comment@noreply.github.com Subject: Re: [stepjam/RLBench] Vision Sensor causes slow steps, and different speeds in general (#53)

pyrep.py:209: UserWarning: Could no t change simulation timestep. You may need to change it to "custom dt" using simulation settings dialog.

Maybe related to this? Basically it’s calling simextstep ~20 times on some of the steps..

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/stepjam/RLBench/issues/53#issuecomment-607479717, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABUSO6RMMJBQA6USWO2PIH3RKORNJANCNFSM4LYJBMZA.

[bycn]

Ah, that was it! Thanks! (Assuming you meant the gripper open/close, after I fixed the action it stopped.)

[bycn]

I think perhaps the velocity should be changed when headless? I think that would make it faster for sims. done = self._robot.gripper.actuate(ee_action, velocity=0.04 => 1)