The stationary vehicle keeps rotating

[JamesZhutheThird]

High Level Description The envision platform seems to use the direction of velocity as the heading of a vehicle. And when the vehicle is stationary, it may look like rotating. Wondering if that's the problem of the dataset (NGSIM) or the platform😵‍💫

[sah-huawei]

Hello @JamesZhutheThird , thanks for the report.

Yes, we are aware of this issue. The problem is due to the way headings are inferred from the NGSIM dataset by our genhistories script -- i.e., the problem is not in the NGSIM dataset itself (NGSIM does not include headings).

We will try to improve on this in future versions. In a past version, our inference algorithm produced smoother, more natural results, but this meant that the position at time t+1 was no longer predictable using basic kinematics from the position and velocity at time t, so we reverted that in order to support a specific type of model training. In the future, we may add an option to allow users such as yourself to control how much smoothing is done when inferring headings.

[Gamenot]

Hi @sah-huawei , I am wondering if we should generate the orientation inference and then keep it separate from the position and velocity of the current t.

[sah-huawei]

Hi @sah-huawei , I am wondering if we should generate the orientation inference and then keep it separate from the position and velocity of the current t.

Well that's kind of what we do already. That is, while position over two successive time-steps is used in order to infer heading (orientation), the computed headings have no impact on the positions we ultimately import.

(Note that at the request of zbzhu99 last summer -- see PR #896, we currently also recompute velocities based on position deltas, as opposed to importing the instantaneous velocities in the dataset, in order to ensure that the speed and heading at each time-step t will precisely predict the position at time t+1. Put another way, we are taking the positions in the NGSIM dataset as accurate / definitive.)

The current problem with our inferred headings is that, in the absence of "instantaneous" headings in the dataset, and without any "smoothing", they are completely dependent on the position deltas across successive timesteps. This is especially problematic in the case of very low vehicle speeds, where the lateral lane position delta in the NGSIM data may be relatively larger relative to the longitudinal lane position delta. Indeed, in the worst case, the delta may be dominated by the lateral position change, leading to an inferred heading that is rotated by nearly 90 degrees relative to the lane direction.

[Note: this would also be affected by the time-step size itself, with larger time-steps potentially leading to worse effects at low speeds, but in the NGSIM dataset at least, the time-step is fixed at 0.1s.]

What we did in the past was to "smooth" our heading inferences by computing them across a wider window of positions (i.e., not just using the position deltas between successive positions, but rather using a sliding window of w positions, where w was roughly 5). The bigger the window size, the more immune our headings are to being overly influenced by lateral movements at slow speed, but also the less likely they are to be predictive of the vehicle's position at time t+1. This was undesirable for an imitation learning model that zbzhu99 was training at the time, so we effectively set w=2. What I am proposing as a solution here is to re-add this window computation, but to make the window size w a parameter that can be specified by the user.

[sah-huawei]

I should also clarify my point about the problem not being in the NGSIM data itself.

It is true that because NGSIM does not include headings, any heading weirdness is due to our inference code. So in that sense, the problem is not in the NGSIM data directly.

However, there are situations where the change in position between time t and t+.1 indicates a very unlikely vehicular motion. For example, there are vehicles for whom, while the lane direction of their lane is in the, say, +x direction, there are samples where from time t to t+.1 where only the y coordinate of the vehicle changes. This is obviously unlikely (you might consider it impossible) and probably points to some inherent error/noise in the positions recorded in the dataset. But we don't currently have a model for that error/noise.

So another alternative we might consider is to simultaneously try to reduce/filter noise in the NGSIM positions while we are computing our headings.

[sah-huawei]

The root of the problem was ultimately noise in the NGSIM dataset. For example, in trajectories-0400-0415.txt in the original data for i80, vehicle 5 has the following sequence of positions:

 t     y       x
319  70.390 372.899
320  70.374 372.899
321  70.388 372.899
322  70.388 372.899
323  70.388 372.899
324  70.388 372.899
325  70.388 372.899
326  70.388 372.899
327  70.388 372.899
328  70.388 372.899
329  70.388 372.899
330  70.388 372.899
331  70.388 372.899
332  70.388 372.899
333  70.388 372.899
334  70.388 372.899
335  70.389 372.899
336  70.388 372.899

Notice that even though the vehicle is traveling in the +x direction throughout its trajectory before and after this, for this stretch of time, its makes no progress in that direction (it is stopped at 372.899) but the y-coordinate is fluctuating slightly (like at timesteps t=319, t=320 and t=335).

This is just one example of the type of "noise" in the NGSIM dataset that can affect heading inferences.

In PR #1219, three different options were added to the dataset spec yaml file to address noise and help "smooth out" heading inferences. Each addresses noise and imprecision issues in the NGSIM dataset in different ways.

If none of these options are used, then headings are inferred from successive positions directly as they are in the dataset (with no filtering or smoothing) and the state at each time-step can be accurately determined solely from the previous time-step, as was requested in PR #896. However, any or all 3 can be used as desired to smooth out headings to prevent sudden orthogonal turns like were seen in the video above.

Here is what an NGSIM yaml file might look like now with comments describing the new options:

trajectory_dataset:                                                                                                                                             
    name: i80_0400-0415
    source: NGSIM
    input_path: ./original/xy-trajectories/i80/trajectories-0400-0415.txt

    # 55mph is roughly 25m/s.  100km/h is roughly 28m/s.
    speed_limit_mps: 28

    flip_y: True
    swap_xy: True

    x_margin_px: 60

    # When inferring headings from positions, each vehicle's angular velocity
    # will be limited to be at most this amount (in rad/sec) to prevent lateral-coordinate
    # noise in the dataset from causing near-instantaneous heading changes.
    # If not specified, then headings are unconstrained.
    max_angular_velocity: 4

    # When inferring headings from positions, a sliding window (moving average)
    # of this size will be used to smooth inferred headings and reduce their dependency
    # on any individual position changes.  Defaults to 2 if not specified.
    heading_inference_window: 5 

    # Speed threshold below which a vehicle's heading is assumed not to change.
    # This is useful to prevent abnormal heading changes that may arise from 
    # noise in position estimates in a trajectory dataset dominating real position
    # changes in situations where the real position changes are very small. 
    # Can use -1 to update headings no matter what the speed.    
    # Defaults to .22 if not specified.   
    heading_inference_min_speed:  .22

    map_net:
        # These are dimensions of the Sumo network in map.net.xml.
        # We map the coordinates in the input dataset onto this space.
        # max_y is required since flip_y is True.
        max_y: 25.02

        # The map width is used to filter position_x values
        # to ensure they're in range.
        width: 310.92

[junluo-huawei]

Guess another layer of filtering in terms of minimal displacement could help.

[sah-huawei]

Guess another layer of filtering in terms of minimal displacement could help.

Well, that's basically the idea behind the new heading_inference_min_speed option. If the displacement isn't big enough for a timestep, then the heading isn't adjusted. (It's just that, by using speed for this, we can take into account how the size of the timestep affects the threshold.)