search

OpenDriveLab / ThinkTwice

[CVPR 2023] Pytorch implementation of ThinkTwice, a SOTA Decoder for End-to-end Autonomous Driving under BEV.
Apache License 2.0
214 stars 21 forks source link

Performance gap in reproducing results #12

Open ShangGL123 opened 1 year ago

ShangGL123 commented 1 year ago

image The reproduced results show that DS is much lower than the numerical value in the paper. Is this result jitter normal?

jiaxiaosong1002 commented 1 year ago

Hi, thanks for your interests.

This codebase has encountered a major refactor and it is mixed with some implementations in our next work (accepted by ICCV 2023 and this time we will release both the original 189K and 2M checkpoints, please stay tuned!). without thorough inspection and tuning due to limited time and computational resource. The model is also retrained.

Thus, there might be some bugs. I check out one of our old team_agent files and write a similar one for this codebase: old_team_agent.py Could you please try this one? (Just replace the original_team_agent) In the meantime, I will cross-check with the private codebase when having time.

Thanks for your time.

Naive-Bayes commented 1 year ago

image The reproduced results show that DS is much lower than the numerical value in the paper. Is this result jitter normal?

Hello, which scenarios file do you use, just the "all_towns_traffic_scenarios_no256.json" provided in the codebase?

raozhongyu commented 8 months ago

Hi, I'd like to ask can you provide the inference time when you evaluate this code. How long it takes ?