search

dotchen / LAV

(CVPR 2022) A minimalist, mapless, end-to-end self-driving stack for joint perception, prediction, planning and control.
https://dotchen.github.io/LAV/
Apache License 2.0
397 stars 68 forks source link

How to evaluate my trained models? #22

Open LifGorg opened 1 year ago

LifGorg commented 1 year ago

Hi Dian, firstly thank you for the strong and beautiful work. I have a slightly dumb question here, which is how can I evaluate my own trained models?

I've done the training steps described in the "training.md", and obtained lidar and uniplanner files in the wandb folder. But I don't see any file named with "routes".

Massive thanks in advance. (this question might be a dumb one, please forgive me)

dotchen commented 1 year ago

Thanks for your interest in the project!

The route files reside in assets folder, you can follow the steps here to evaluate the agents. Let me know if you run into any issues.

JianLiMech commented 1 year ago

Hello, have you done evaluation? I did ROUTES=leaderboard/data/routes_valid.xml ./leaderboard/scripts/run_evaluation.sh but I had the following problem.

pygame 2.1.2 (SDL 2.0.16, Python 3.7.10)
Hello from the pygame community. https://www.pygame.org/contribute.html
usage: leaderboard_evaluator.py [-h] [--host HOST] [--port PORT]
                                [--trafficManagerPort TRAFFICMANAGERPORT]
                                [--trafficManagerSeed TRAFFICMANAGERSEED]
                                [--debug DEBUG] [--record RECORD]
                                [--timeout TIMEOUT] --routes ROUTES
                                --scenarios SCENARIOS
                                [--repetitions REPETITIONS] -a AGENT
                                [--agent-config AGENT_CONFIG] [--track TRACK]
                                [--resume RESUME] [--checkpoint CHECKPOINT]
leaderboard_evaluator.py: error: unrecognized arguments: --route-id=

Do you know how can I evaluate? Thanks in advance!