Closed YangYangLeo closed 1 year ago
Hi @patk-motional ,
when I submit as you said, the new issue comes out.Just like follow, do you have any ideas about that?
I am really thankful.
Traceback (most recent call last):
File "/opt/conda/envs/nuplan/lib/python3.9/site-packages/grpc/_server.py", line 443, in _call_behavior
response_or_iterator = behavior(argument, context)
File "/nuplan_devkit/nuplan/submission/challenge_servicers.py", line 98, in InitializePlanner
planners = build_planners(self._planner_config, None)
File "/nuplan_devkit/nuplan/planning/script/builders/planner_builder.py", line 58, in build_planners
return [_build_planner(planner, scenario) for planner in planner_cfg.values()]
File "/nuplan_devkit/nuplan/planning/script/builders/planner_builder.py", line 58, in
You can try a few more methods:
Revert ml_planner.yaml to the original. It should look like this:
ml_planner:
_target_: nuplan.planning.simulation.planner.ml_planner.ml_planner.MLPlanner
_convert_: 'all'
model_config: ??? # Dictionary key from existing planner model config (e.g. raster_model)
checkpoint_path: ??? # Path to trained model checkpoint
conda run -n nuplan --no-capture-output python -u nuplan/planning/script/run_submission_planner.py output_dir=/tmp/ model=YOUR_MODEL planner=ml_planner planner.ml_planner.model_config=\${model} planner.ml_planner.checkpoint_path="YOUR_CHECKPOINT"
replace YOUR_PLANNER
and YOUR_CHECKPOINT
.
Modify ml_planner.yaml. It should look like this:
ml_planner:
_target_: nuplan.planning.simulation.planner.ml_planner.ml_planner.MLPlanner
_convert_: 'all'
model_config: ??? # Dictionary key from existing planner model config (e.g. raster_model)
checkpoint_path: "YOUR_CHECKPOINT" # Path to trained model checkpoint
conda run -n nuplan --no-capture-output python -u nuplan/planning/script/run_submission_planner.py output_dir=/tmp/ model=YOUR_MODEL planner=ml_planner planner.ml_planner.model_config=\${model}
replace YOUR_PLANNER
and YOUR_CHECKPOINT
.
Modify default_common.py
# Default common configs
defaults:
# Add common items
- scenario_builder: nuplan_mini
- scenario_filter: one_continuous_log
- model: YOUR_MODEL
# Worker that is used to run simulations
- worker: ray_distributed
Modify ml_planner.yaml to the original. It should look like this:
ml_planner:
_target_: nuplan.planning.simulation.planner.ml_planner.ml_planner.MLPlanner
_convert_: 'all'
model_config: ${model} # Dictionary key from existing planner model config (e.g. raster_model)
checkpoint_path: "YOUR_CHECKPOINT" # Path to trained model checkpoint
conda run -n nuplan --no-capture-output python -u nuplan/planning/script/run_submission_planner.py output_dir=/tmp/ planner=ml_planner
replace YOUR_PLANNER
and YOUR_CHECKPOINT
.
HI @patk-motional, I have tried all 3 ways you have mentioned above. Way 1, I got the error "hydra.errors.InstantiationException: Top level config has to be OmegaConf DictConfig, plain dict, or a Structured Config class or instance"
Way2 and Way3, I got the error "FileNotFoundError: [Errno 2] No such file or directory: '/nuplan-devkit/best.ckpt'" However, I have double checked that my model(best.ckpt) is under /nuplan-devkit for the container test-submission.
Do you have any more ideas?
Thanks a lot.
@patk-motional , all the docker images (3 ways above) run locally present as follow,
Is that run successfull in local?
But when it is submitted, it comes out issue.
No, you should see this at the end
nuplan-devkit-simulation-1 | 2023-05-23 05:26:18,306 INFO {/nuplan_devkit/nuplan/planning/simulation/runner/executor.py:127} Number of successful simulations: 1
nuplan-devkit-simulation-1 | 2023-05-23 05:26:18,306 INFO {/nuplan_devkit/nuplan/planning/simulation/runner/executor.py:128} Number of failed simulations: 0
nuplan-devkit-simulation-1 | 2023-05-23 05:26:18,306 INFO {/nuplan_devkit/nuplan/planning/script/utils.py:147} Finished executing runners!
nuplan-devkit-simulation-1 | 2023-05-23 05:26:18,330 INFO {/nuplan_devkit/nuplan/planning/script/utils.py:74} Saved runner reports to /data/exp/nuplan/exp/simulation/open_loop_boxes/2023.05.23.05.26.09/runner_report.parquet
nuplan-devkit-simulation-1 | 2023-05-23 05:26:18,330 INFO {/nuplan_devkit/nuplan/planning/simulation/main_callback/time_callback.py:27} Simulation duration: 00:00:06 [HH:MM:SS]
nuplan-devkit-simulation-1 | 2023-05-23 05:26:18,357 INFO {/nuplan_devkit/nuplan/planning/simulation/main_callback/metric_file_callback.py:79} Metric files integration: 00:00:00 [HH:MM:SS]
nuplan-devkit-simulation-1 | 2023-05-23 05:26:18,392 INFO {/nuplan_devkit/nuplan/planning/simulation/main_callback/metric_aggregator_callback.py:58} Running metric aggregator: open_loop_boxes_weighted_average
nuplan-devkit-simulation-1 | 2023-05-23 05:26:18,398 INFO {/nuplan_devkit/nuplan/planning/simulation/main_callback/metric_aggregator_callback.py:69} Metric aggregator: 00:00:00 [HH:MM:SS]
Rendering histograms: 100%|██████████| 6/6 [00:01<00:00, 5.41it/s]
nuplan-devkit-simulation-1 | 2023-05-23 05:26:20,220 INFO {/nuplan_devkit/nuplan/planning/simulation/main_callback/metric_summary_callback.py:344} Metric summary: 00:00:01 [HH:MM:SS]
nuplan-devkit-simulation-1 | 2023-05-23 05:26:20,220 INFO {/nuplan_devkit/nuplan/planning/script/run_simulation.py:80} Finished running simulation!
In https://github.com/motional/nuplan-devkit/issues/302#issuecomment-1549319730 what is YOUR_MODEL, where should this be set and where should model be set so that ${model} is replaced properly?
YOUR_MODEL
is a yaml file. You should only need to replace YOUR_MODEL
. There is no need to modify anything else. Note this is assuming you are using ml_planner.yaml
YOUR_MODEL
is a yaml file. You should only need to replaceYOUR_MODEL
. There is no need to modify anything else. Note this is assuming you are usingml_planner.yaml
Thanks.
As suggested in https://github.com/motional/nuplan-devkit/issues/302#issuecomment-1549319730 none of the three options work. requesting the developers to look into this. We are unable to run simulation planner on local copies.
Hi @sindhu-pr, it is hard for us to help if you are unable to run it locally. Have you tried with the example planner yet?
Hello, Could you please tell me what should I write for "model_config“ if I want to submit my own ml_planner?
I just type to "go_model" like the picture above.
But I got "validation_challenge99.log 2023-05-13 14:15:39,072 : ERROR : Planner initialization failed! 2023-05-13 14:15:44,421 : ERROR : Planner initialization failed!" issue after submitting image.
I am wondering if this issue is related to the"model_config" ? If so, could you please tell me a suggestion?Thanks a lot.
BTW here is my id: {"submitted_image_uri": "937891341272.dkr.ecr.us-east-1.amazonaws.com/nuplan-planning-challenge-1856-participant-team-20954:83cee015-17ba-4524-9e75-0ebcfedeb3a2"}