search

autonomousvision / carla_garage

[ICCV'23] Hidden Biases of End-to-End Driving Models
MIT License
203 stars 16 forks source link

About reproducing perception PlanT #5

Closed Reezzz702 closed 1 year ago

Reezzz702 commented 1 year ago

Hi, love the work from your team.

I would like to re-train perception PlanT from the codebased, however I found that setting "use_plant = True" in config.py can only train PlanT without perception part. Are there something that I missed?

Kait0 commented 1 year ago

You are not missing anything, perception PlanT is not implemented in this repository. I used PlanT mostly for quick prototyping of output representations, so I only needed the privileged part.

If you want to run a perception PlanT you need to implement it yourself. If you want to run something similar to perception PlanT, you only need to implement an agent file. TF++ predicts all the inputs that PlanT needs so they can be stitched together. You need to re-implement the bounding box tracking from the PlanT repo to extract vehicle speeds and convert the predicted traffic light / stop sign bounding boxes to flags.

If you happen to do that, you can also make a pull request, would probably be useful for others as well.

Reezzz702 commented 1 year ago

Thanks for your reply.