Kait0
commented
1 year ago Yes, as mentioned in the appendix the driving score of TF++ Wp is 75 on longest6. To be more precise the numbers are: DS 75 RC 97 IS 0.76
This is similar to TF++ which had between 72-74 DS (table 6 and 13) on longest6.
The numbers of TF++ WP Ensemble are: DS 77 RC 97 IS 0.79
I do not know why there is such a big gap between the two representations on the CARLA leaderboard. That effect is not reproducible on the publicly available towns.
However, in practice, a vehicle's steering angle is highly influenced by its speed, particularly at higher speeds.
This is not really a problem at the speeds models are driving in CARLA v1 (e.g. < 30 km/h) To generalize this approach to higher driving speeds one would probably need a more sophisticated controller (than the simple PID I am using here) that takes into account the vehicles speed.
SmashChen
Hi authors, i would like to discuss with you about the two output representations of TF++ and TF++ WP. TF++ presents a novel approach to transforming the output representations of path and target speed into control signals. It solves the problem that the steering value disappears when the vehicle is stopped. I observed that in the Longest6 benchmark, vehicles often stop at corners due to heavy traffic.
In my understanding, the path prediction remains constant, even though the traffic conditions vary. This implies that at each location on the map, the vehicle maintains a fixed steering angle, unaffected by the target speed. However, in practice, a vehicle's steering angle is highly influenced by its speed, particularly at higher speeds. During training and evaluation, the agent encounters diverse traffic conditions, each associated with distinct target speeds.
I noticed that in your paper, you compared the performance of TF++WP and TF++ methods on the LAV benchmark and obtained similar results (49DS and 50DS). As shown in Table 3 in the paper:
At the same time, I also noticed that in the CARLA leaderboard benchmark, there is a big gap between TF++ and TF++ WP (TF++ DS 52.816, TF++WP DS 61.570, TF++WP Ensemble DS 66.317).
Is it normal for these two methods to have wildly different results on different benchmarks? Also, I am very interested in whether you have evaluated TF++WP on the Longest6 benchmark, and if so, can you tell me the results?
Thank you very much for your exceptional work!