search

autonomousvision / transfuser

[PAMI'23] TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving; [CVPR'21] Multi-Modal Fusion Transformer for End-to-End Autonomous Driving
MIT License
1.04k stars 175 forks source link

A data generation problem #40

Closed Halcyon-Lexie closed 2 years ago

Halcyon-Lexie commented 2 years ago

Hi, when I use the autopilot to start data generation, I get a "failure" as the following picture: ![Uploading 2021-12-23 19-24-01屏幕截图.png…]() I know this problem is common, but I worry it will affect the final result of model training since we imitate an imperfect expert. How do you think about this?

Halcyon-Lexie commented 2 years ago

2021-12-23 19-24-01屏幕截图

Halcyon-Lexie commented 2 years ago

In addition, I don't why I get some black points in the images. Do you know the cause? 0053

ap229997 commented 2 years ago

Hi, sorry for the late reply.

The expert uses collision avoidance heuristics which has certain loopholes resulting in few collisions. Ideally, this should be removed from the training data but I think as long as this type of data is very less compared to the proper driving data, the model should be able to learn proper driving behavior.

These black dots are due to some rendering issues when running CARLA with opengl. You can try running with vulkan to remove these artifacts. For running with vulkan, just replace -opengl with -vulkan when launching the CARLA server.