Question about transfuser++ dataset (train & validation)

[yihang99]

Hello, thank you for your great work first. I'd like to ask the following question about your latest paper transfuser++:

In table6 and table7 you mentioned that the longset6 is used as training towns and LAV is used as validation. Does that mean that you collected the expert data (for imitation) in routes of longest6?

Also, in appendix C I see that 'we test our additions to TransFuser by repeating the experiments on the training towns (Longest6) while training with all data. ' I wonder what does 'all data' mean here, comparing to the dada you used before.

Shortly, I am kind of confused by the category of the data you used (By town or by route?). I would be greatful if you can make it clearer to me. : )

[Kait0]

The training routes are different from the evaluation routes, you can find them here. These are the same routes as used by TransFuser. The training routes are much more routes, that are short and contain 1 scenario each.

What we refer to as training and validation is w.r.t. the towns. So in longest6 we use data collected from all Towns for training, whereas for the LAV routes the data collected in Town 02 and Town 05 is not used during training (because these are the towns where the LAV evaluation takes place).

Longest6 evaluates many routes in town 1-6 so it has more diversity than the LAV routes, but since it uses most of the towns for evaluation you can't withhold them during training. You can think of Longest6 measuring Level 4 Driving performance and LAV measuring Level 5 driving performance (as in generalization to new locations is also measured).

[yihang99]

Thank you for your kindly reply! I have 2 more simple questions about your experiments if you don't mind:

1. When I read & reproducing your code, I found that in the _route_scenariolocal.py file where you set the number of other NPCs in the map, only the number of vehicles are set (to 500 for evaluation), so that no pedestrians occur in evaluation. Is that a standard setting for longest6 benchmark? (Or did I understand sth wrong about this?)

2. Have you tried the version without lidar sensor (like latentTF in the old transfuser)? I am curious about the results if you have tried the related experiments : )

Thank you again for your time spent on my questions~

[Kait0]

1. No pedestrians is the standard setting in any carla leaderboard based benchmark. Pedestrians only occur as part of scenario 3. 500 vehicles (which just sets the vehicles to the maximum per town) is only used in longest6, other benchmarks like LAV use lower traffic densities.

2. I have not tried Latent TransFuser in this project. We did reproduce the original AIM from the first TransFuser paper in Appendix C.5, which does not use LiDAR.

[yihang99]

IC, thanks for your reply!