OpenDriveLab / Vista

[NeurIPS 2024] A Generalizable World Model for Autonomous Driving
https://opendrivelab.com/Vista
Apache License 2.0
580 stars 43 forks source link

Have you considered putting it in carla for closed-loop testing? #19

Open hunkyu opened 4 months ago

hunkyu commented 4 months ago

Have you considered putting it in carla for closed-loop testing?

Little-Podi commented 4 months ago

Hi, thanks for this good question. We haven't tried this yet for two reasons. First, our model is targeting at real-world scenarios, while CARLA is a simulated world with different appearances and rules. Second, although I'm not an expert of CARLA, I think that CARLA itself is a perfect world model of its own scenes. In other words, mimicing a visual world model upon CARLA might be repetitive and meaningless as CARLA already provides the desired function, i.e. getting the future status.

hunkyu commented 4 months ago

@Little-Podi I agree. If we can see the generalization effect in Carla, it is a very good work. If we assume that the world model is good enough, is it possible to achieve SOTA level in the closed-loop performance of Carla?