Scalability Requirements for Agents

[sah-huawei]

We've received a request from @zbzhu99 and his associated team for SMARTS to support the simultaneous hijacking of a great many (over 1000) vehicles from a traffic history dataset (NGSIM) for control with a model (potentially trained using Imitation Learning), perhaps in a manner similar to what is done in our [examples/history_vehicles_replacement_for_imitation_learning.py] example.

This issue aims to clarify the requirements of their use case. Below are some questions whose answers will help us to determine the scope of the work.

• In general, what features of SMARTS motivate its use in this research?
  • What degree of realism is required?
• What "agent interface" is desired for the hijacked vehicles?
  • Note that the agent interface specifies the sensors attached to each vehicle (including cameras, neighborhood vehicle detection, accelerometer, waypoints, etc.) and the "action space" that the agent can use. Available action spaces include "continuous" (throttle, steering, brakes), "imitation" (acceleration, angular velocity) and "trajectory", among others.
• Which aspects of Physics need to be simulated by SMARTS? For example:
  • Does SMARTS need to detect collisions among the vehicles?
  • Does SMARTS need to model the inertial properties of the vehicles? (so as to constrain motions to things that are physically possible for the vehicle)
  • Does SMARTS need to model traction/friction of tires?
• Related: What type(s) of chassis are desired for the vehicles?
  • Options include Ackermann and Box.
  • An Ackermann chassis uses an articulated vehicle model that includes joints and mass and supports much more realistic physical modeling (e.g., tire traction, application of forces, etc.). This is used by the "continuous" action space.
  • A Box chassis is primitive, just occupying the space of the vehicle for collision detection. This is used by the "imitation" and "trajectory" action spaces.
  • Note: SMARTS currently only supports only one vehicle model for Ackermann chassis, which assumes a specific mass and dimensions (for an "average" vehicle). Traffic history datasets like NGSIM do not include the real masses of the vehicles that generated the data, so this is an approximation. New physical models would need to be created to support Trucks and/or Cycles in history datasets.
• Is it necessary to support hijacking Trucks or Cycles in addition to Cars in the (NGSIM) dataset?
• What UI (Envision) features are needed for evaluating the simulation/model? For example:
  • the ability to watch the simulation in real-time?
  • the ability to record a simulation run and watch it ("as if replaying a video")
    • the ability to seek to specific times in the replay ("scrub"), both in the future and in the past?
    • recording conditional on simulation state (e.g. around when a collision occurs?)

[sah-huawei]

I got this reply via email from @zbzhu99:

Thank you for dealing with our request. First I want to clarify that in NGSIM I-80 dataset, there are at most 200-300 vehicles simultaneously appear on the road. Hijacking over 1000 vehicles is not needed for now. For detailed requirements, in general we do not need complex physical modeling. • For the observation, neighborhood vehicle detection and ego speed, position, heading is needed. Visual information from cameras is optional. The "imitation" action space is needed. • Collision detection is required. I do not know the specific details about the inertial properties of the vehicles and how they influence the control process in SMARTS, so that I am not sure of whether they are needed. Traction/friction of tires is unnecessary. • Box chassis is okay. • We want to hijack Trucks or Cycles in the (NGSIM) dataset, and they should have different size when we control them. • For envision replay, the ability to record a simulation run and watch it is required. The ability of real-time replay is preferred. I don't quite understand what conditional on simulation state means, and maybe you can explain in more detail?

[sah-huawei]

Note: given that they are using the Imitation action space, they are already using Box chassis (and so there won't be substantial additional improvements by bypassing pybullet -- there may be still be some, so we should experiment with it).

We will need to add the ability to hijack cycles and trucks -- we don't currently support this. Issue #1369 will track this.

Issues #1356, #1361 and #1366 track what is needed for Envision.