search

jchengai / planTF

[ICRA'2024] Rethinking Imitation-based Planner for Autonomous Driving
https://jchengai.github.io/planTF/
208 stars 15 forks source link

Failed Simulation #1

Closed anubhavjetley2424 closed 11 months ago

anubhavjetley2424 commented 1 year ago

Hi thanks for posting this, I am trying to run a simulation using the pre-trained planTF model. The code seems to be working but i get Failed simulations [log, token]: [2021.06.09.17.23.18_veh-38_02526_03027, 267e4438ffd856d7] [2021.06.08.16.31.33_veh-38_01589_02072, 4d53af4085a054a3] [2021.06.07.12.54.00_veh-35_01843_02314, dca79525102b5cec] [2021.05.12.23.36.44_veh-35_00152_00504, 38b71d6e2dc65a1d] [2021.06.09.17.23.18_veh-38_02526_03027, 28e28684a2045c94] [2021.05.12.23.36.44_veh-35_00152_00504, 00699cdbd1a051bd] [2021.06.14.16.48.02_veh-12_04057_04438, dadfa9349da658fa] [2021.07.24.20.37.45_veh-17_00015_00375, b2a7b4c83d275e56] [2021.10.05.07.10.04_veh-52_01442_01802, 7fbb13b94d3859b1] [2021.10.01.19.16.42_veh-28_02011_02410, ce1d2b73af5752fd] [2021.10.06.07.26.10_veh-52_00006_00398, 5840c722110058c3] [2021.06.09.14.03.17_veh-12_02584_02970, 75acdc405ea45c2b] [2021.10.05.07.10.04_veh-52_01442_01802, c037d6199bb25375] [2021.10.06.07.26.10_veh-52_00006_00398, f923613ac41355b1] [2021.06.09.14.58.55_veh-35_01894_02311, 32c0dd98fbcf59c0] [2021.06.09.17.23.18_veh-38_02526_03027, 6913493fd8a05e04] [2021.06.08.12.54.54_veh-26_04262_04732, e4d09605e2b6563e] [2021.10.05.07.10.04_veh-52_01442_01802, 7aa620b613825b60] [2021.06.03.13.55.17_veh-35_00073_00426, c537d837b0585c30] [2021.06.14.19.22.11_veh-38_01480_01860, 2813646573e05ee4] [2021.05.12.22.28.35_veh-35_00620_01164, 59911f1c671d5e9a] [2021.06.03.13.55.17_veh-35_00073_00426, f2877ddb99735b4d] [2021.06.08.16.31.33_veh-38_01589_02072, e7f5e3281cd85e35] [2021.06.09.17.37.09_veh-12_00404_00864, de3704cea95b5789] [2021.05.12.23.36.44_veh-35_00152_00504, 2060219b87b054ec] [2021.06.28.16.57.59_veh-26_00016_00484, 34ebb72887ac5e14] [2021.06.14.19.22.11_veh-38_01480_01860, e1c44502ea035100] [2021.06.09.17.23.18_veh-38_02526_03027, 6544d3038e325081] [2021.08.09.17.55.59_veh-28_00021_00307, c0315fb66af05f2e] [2021.06.14.19.22.11_veh-38_01480_01860, 1dbcee5b8ba55210] [2021.06.28.16.29.11_veh-38_03263_03766, 4a7ba062aaa45c57] [2021.06.28.16.29.11_veh-38_01415_01821, 195890475ff95098] [2021.06.28.16.29.11_veh-38_01415_01821, f3071ad83f015577] [2021.07.16.20.45.29_veh-35_00600_01084, 83578a1c62bd54d9] [2021.06.14.16.48.02_veh-12_04057_04438, 0f8b62959c995e7d] [2021.06.14.17.26.26_veh-38_04544_04920, 25337666152a5461] [2021.10.01.19.16.42_veh-28_03307_03808, f6239f635a435bb4] [2021.07.16.18.06.21_veh-38_03231_03712, 8f13eaa464725248] [2021.06.09.14.58.55_veh-35_01095_01484, 24624c460bc45af7] [2021.07.16.00.51.05_veh-17_01352_01901, 8139656ffee557e0] [2021.05.12.23.36.44_veh-35_02035_02387, ecc476fc2c51540b] [2021.08.30.14.54.34_veh-40_00439_00835, 8008f385038251bf] [2021.06.28.15.02.02_veh-38_02398_02848, d9cbb4ef4b8f5f20] [2021.06.09.17.37.09_veh-12_00404_00864, c0605bfd21bf5beb] [2021.05.12.22.28.35_veh-35_00620_01164, 39dce73ca61f5409] [2021.07.16.18.06.21_veh-38_04933_05307, 190aaf425cf85ef6] [2021.08.30.14.54.34_veh-40_00439_00835, 3cb701313ff0518e] [2021.10.01.19.16.42_veh-28_02011_02410, a3b4b8e47696534e] [2021.07.16.18.06.21_veh-38_04471_04922, a136e73ce6125eea] [2021.05.12.22.00.38_veh-35_01008_01518, 90b3eef10e6457fc] [2021.06.14.17.26.26_veh-38_04544_04920, b391c0fa8349515b] [2021.06.09.14.58.55_veh-35_01894_02311, 9e23b455e95e597e] [2021.06.23.17.31.36_veh-16_00016_00377, f6936f339a0355f1] [2021.05.12.23.36.44_veh-35_01133_01535, 05941ddf98795109] [2021.06.09.14.03.17_veh-12_02584_02970, 10460185498255a9] [2021.06.07.12.54.00_veh-35_01843_02314, 7560dff8b9e056ec] [2021.07.16.18.19.22_veh-35_00440_00858, 689d7276979d53d1] [2021.06.09.17.23.18_veh-38_02526_03027, c70dd6165bbf5c4e] [2021.10.06.17.43.07_veh-28_00508_00877, 9ba66efce8d554bc] [2021.06.14.17.26.26_veh-38_04544_04920, 6939f1a2a6d65af1] [2021.06.28.16.57.59_veh-26_00016_00484, 629d33bbb1675557] [2021.06.09.12.39.51_veh-26_05620_06003, 85588b27db4c5538] [2021.05.12.23.36.44_veh-35_00152_00504, fbd4ceba703c53e3] [2021.06.28.15.02.02_veh-38_02398_02848, c3d9a68c73845fca] [2021.06.03.13.55.17_veh-35_00073_00426, 2c33422b474a5c2e] [2021.06.09.12.39.51_veh-26_05620_06003, 24fffd68b8175e25] [2021.06.28.15.02.02_veh-38_02398_02848, 1bd1b0dd2b585b1d] [2021.07.16.00.51.05_veh-17_01352_01901, 86a2d82d41d95e0b] [2021.08.17.17.17.01_veh-45_02314_02798, d2319709821e5022] [2021.06.09.14.03.17_veh-12_02584_02970, 3887c7bf925c59dd] [2021.07.16.18.06.21_veh-38_04471_04922, 7d4b75bf75345aa1] [2021.06.08.12.54.54_veh-26_04262_04732, 4caf1c76e0bc5a1e] [2021.05.12.23.36.44_veh-35_02035_02387, d9ead70fa8a45372] [2021.07.16.18.06.21_veh-38_04471_04922, 607df244349d5cbc] [2021.10.05.07.10.04_veh-52_01442_01802, 0d48c024a6455ace] [2021.08.17.17.17.01_veh-45_02314_02798, 52a774425b6c5fc2] [2021.08.17.17.17.01_veh-45_02314_02798, 3bac5cd7758e5a67] [2021.06.09.14.58.55_veh-35_01894_02311, 7444f1db4f6f525d] [2021.06.09.12.39.51_veh-26_05620_06003, f041e9875d3d5a02] [2021.06.03.13.55.17_veh-35_00073_00426, a9e1ec5f46425ca3] [2021.06.09.14.58.55_veh-35_01894_02311, 25e25240711d5a96] [2021.06.09.14.58.55_veh-35_01894_02311, d6ba414b484f5c81] [2021.06.23.15.56.12_veh-16_00839_01285, 71eaa46d8ab9551b] [2021.06.08.14.35.24_veh-26_02555_03004, 207539921f1c5724] [2021.10.01.19.16.42_veh-28_03307_03808, 79e6a471ed195832] [2021.07.16.18.06.21_veh-38_04933_05307, dc917efde6a05ffb] [2021.07.16.00.51.05_veh-17_01352_01901, dd2b9f8231bd5751] [2021.06.09.17.23.18_veh-38_02526_03027, af4f6494222456ee] [2021.07.24.20.37.45_veh-17_00015_00375, 9b82d586d2ff54f6] [2021.06.23.15.56.12_veh-16_00839_01285, f6f1121a1bd051fe] [2021.07.16.18.06.21_veh-38_04933_05307, cb777ba786075d3f] [2021.10.11.02.57.41_veh-50_01522_02088, 86781f95ba6a5ded] [2021.06.23.17.31.36_veh-16_00016_00377, ccc6e6eed4e2553d] [2021.06.09.14.58.55_veh-35_01894_02311, deb79026c30054fe] [2021.06.14.17.26.26_veh-38_04544_04920, 4babfbb173225325] [2021.06.23.17.31.36_veh-16_00016_00377, 522a1df712615880] [2021.06.23.16.54.19_veh-35_00808_01256, b4ca4297844e5fe2] [2021.06.09.14.58.55_veh-35_01894_02311, 61e7b925dec958cd] [2021.06.28.16.57.59_veh-26_00016_00484, be79edfdf1eb5e80] [2021.06.08.14.35.24_veh-26_02555_03004, eb1761280ff8521f]

In the nuboard log.txt section the following error occurs. Error(s) in loading state_dict for PlanningModel: Unexpected key(s) in state_dict: "agent_encoder.history_encoder.levels.0.blocks.0.attn.rpb", "agent_encoder.history_encoder.levels.0.blocks.0.attn.qkv.weight", "agent_encoder.history_encoder.levels.0.blocks.0.attn.qkv.bias", "agent_encoder.history_encoder.levels.0.blocks.0.attn.proj.weight", "agent_encoder.history_encoder.levels.0.blocks.0.attn.proj.bias", "agent_encoder.history_encoder.levels.0.blocks.1.attn.rpb", "agent_encoder.history_encoder.levels.0.blocks.1.attn.qkv.weight", "agent_encoder.history_encoder.levels.0.blocks.1.attn.qkv.bias", "agent_encoder.history_encoder.levels.0.blocks.1.attn.proj.weight", "agent_encoder.history_encoder.levels.0.blocks.1.attn.proj.bias", "agent_encoder.history_encoder.levels.1.blocks.0.attn.rpb", "agent_encoder.history_encoder.levels.1.blocks.0.attn.qkv.weight", "agent_encoder.history_encoder.levels.1.blocks.0.attn.qkv.bias", "agent_encoder.history_encoder.levels.1.blocks.0.attn.proj.weight", "agent_encoder.history_encoder.levels.1.blocks.0.attn.proj.bias", "agent_encoder.history_encoder.levels.1.blocks.1.attn.rpb", "agent_encoder.history_encoder.levels.1.blocks.1.attn.qkv.weight", "agent_encoder.history_encoder.levels.1.blocks.1.attn.qkv.bias", "agent_encoder.history_encoder.levels.1.blocks.1.attn.proj.weight", "agent_encoder.history_encoder.levels.1.blocks.1.attn.proj.bias", "agent_encoder.history_encoder.levels.2.blocks.0.attn.rpb", "agent_encoder.history_encoder.levels.2.blocks.0.attn.qkv.weight", "agent_encoder.history_encoder.levels.2.blocks.0.attn.qkv.bias", "agent_encoder.history_encoder.levels.2.blocks.0.attn.proj.weight", "agent_encoder.history_encoder.levels.2.blocks.0.attn.proj.bias", "agent_encoder.history_encoder.levels.2.blocks.1.attn.rpb", "agent_encoder.history_encoder.levels.2.blocks.1.attn.qkv.weight", "agent_encoder.history_encoder.levels.2.blocks.1.attn.qkv.bias", "agent_encoder.history_encoder.levels.2.blocks.1.attn.proj.weight", "agent_encoder.history_encoder.levels.2.blocks.1.attn.proj.bias". im running .sh ./script/plantf_single_scenarios.sh

I changed the script for this to this: image

Thanks in advance any help will be very much appreciated.

jchengai commented 1 year ago

Hi @anubhavjetley2424 , try to use this scenario filter config for the nuplan mini split.

_target_: nuplan.planning.scenario_builder.scenario_filter.ScenarioFilter
_convert_: 'all'

scenario_types: null                # List of scenario types to include
scenario_tokens: null               # List of scenario tokens to include

log_names: null                     # Filter scenarios by log names
map_names: null                     # Filter scenarios by map names

num_scenarios_per_type: null        # Number of scenarios per type
limit_total_scenarios: 10         # Limit total scenarios (float = fraction, int = num) - this filter can be applied on top of num_scenarios_per_type
timestamp_threshold_s: null         # Filter scenarios to ensure scenarios have more than `timestamp_threshold_s` seconds between their initial lidar timestamps
ego_displacement_minimum_m: null    # Whether to remove scenarios where the ego moves less than a certain amount
ego_start_speed_threshold: null     # Limit to scenarios where the ego reaches a certain speed from below
ego_stop_speed_threshold: null      # Limit to scenarios where the ego reaches a certain speed from above
speed_noise_tolerance: null         # Value at or below which a speed change between two timepoints should be ignored as noise.

expand_scenarios: false             # Whether to expand multi-sample scenarios to multiple single-sample scenarios
remove_invalid_goals: false         # Whether to remove scenarios where the mission goal is invalid
shuffle: false                      # Wheth
zhangqiangbing commented 3 months ago

@anubhavjetley2424 hi i also get the same problem. First time i run the original script plantf_single_scenarios.sh. Secondly, i copy your modified script and use the scenario filter config by jchengai. They both failed. The error are same: RuntimeError: Error(s) in loading state_dict for PlanningModel: Unexpected key(s) in state_dict: "agent_encoder.history_encoder.levels.0.blocks.0.attn.rpb" It seems the mode in code and the one in planTF.ckpt not match. strange

Have you resolve the bug?

kushal2000 commented 1 month ago

I have the same doubt as above.