Manually control simulation step?

[gkahn13]

Is it possible to manually control when and for how long the simulator steps? Basically I would like to use AirSim similar to OpenAI Gym environments in which I get the current observation, compute a desired action, and then execute the action and step the environment.

I noticed simple_flight uses the SteppableClock, but I'm unclear if SteppableClock is what I need, and if so, how to use it with the Python/C++ API.

[sytelus]

Yes, simple_flight uses simulated clock that can be stepped. However this functionality is not exposed via APIs yet. It is probably not too much work to add this. However, I'd say you probably want to avoid using this mode. In real world, the world does not freeze while you make computation. The latency in observation and action are integral part of reality.

[gkahn13]

Agreed you don't want this for final real-world testing, but there are some advantages: (1) Reproducibility. This is a continuing problem in deep RL (e.g. paper on it) and a steppable environment with fixed random seeds helps. (2) Super real-time experiments. Plus you could do a steppable environment and still have repeatable stochasticity (latency, noise, etc), but that would require a bit more work.

Hope this gets added to the API soon!

[piezox]

+1, gkahn13 ! Possibility of steppable environments is important for modeling & simulation in general (see Simulink reference for example)

[jtatusko]

Yes, simple_flight uses simulated clock that can be stepped. However this functionality is not exposed via APIs yet. It is probably not too much work to add this. However, I'd say you probably want to avoid using this mode. In real world, the world does not freeze while you make computation. The latency in observation and action are integral part of reality.

Unfortunately this complicates training RL algorithms such as Actor Critic (with rollouts). Even if you're using a ML-controlled drone in the real world, you would only run inference. Inference latency will be much lower than training latency, making the time-delay less relevant. It's also useful to abstract away the clock. I'm looking to train an agent on 500+ environment instances in parallel (and so will other researchers). Running more in parallel is going to increase compute latency so I need to adjust the sim speed relative to how many environments I'm running. This seems like a bit too much of a burden

Is there a link to the relevant code for "simple_flight"? If it's not too difficult, maybe I could contribute this

Edit: nevermind, I see the step sim pausing API

[ghost]

@jtatusko Could you elaborate on the step sim pausing API and how you used it in your project? I'm working with RL algorithms as well and stepping through the environment would be a desirable feature. (I'm using the car interface)

[YumaTheCompanion]

@MoritzGa If I'm understanding correctly, you are looking for a way to pause simulation? In my case,

client.simPause(true); // To pause and client.simPause(false); // To continue

works well enough.

I don't know of a stepping API but maybe using that and a thread, it's possible to step through the simulation.

[ghost]

@YumaTheCompanion Yes, that is kind of what I was looking for. To be more specific I want to take an observation in the form of raw image coming from the front cam, perform an inference step to get the proposed action and let the environment run for lets say 10 ms before taking the next observation. Otherwise my main script is way to fast and multiple actions cancel eachother without the vehicle able to move.

[jtatusko]

@MoritzGa

I'm writing a gym-like wrapper around airsim. Step method would look roughly like this:

def step(action):
  client.setCarControls(action)
  client.continueForTime(1)
  state = client.getCarState()
  return state

Would ideally like to run simulator clock much faster than real-time so that continueForTime would advance simulation time by one second but take minimal time to compute the next state. Unfortunately haven't been able to get the sim clock to go faster than real-time for vehicles, see this issue: https://github.com/microsoft/AirSim/issues/2163

[ghost]

@jtatusko

Thanks for your reply! Is the continueForTime(delta) API actually working? See issue: #1719 I'm using delta=0.5 and actions are changig way faster than expected.

[jtatusko]

@MoritzGa

Haven't implemented anything yet. I'll post if I find workarounds

[Aadi0902]

@jtatusko Were you able to find a way to implement a steppable clock in airsim instead of the wall clock that it uses currently?