Question when training on full dataset: database disk image is malformed

[duanyiqun]

Hi Motional team, appreciated a lot for the great work.

I've successfully run the tutorial and customized training on the nuplan-mini dataset.

However, when I changed the scenario builder to nuplan for training on the full data. I continuously get error like below:

RuntimeError: ray::wrapped_fn() (pid=105316, ip=192.168.30.202)
  File "/workspace/code/autonomous-driving-project/nuplan-devkit/nuplan/planning/utils/multithreading/ray_execution.py", line 59, in wrapped_fn
    result = fn(*args, **kwargs)
  File "/workspace/code/autonomous-driving-project/nuplan-devkit/nuplan/planning/scenario_builder/nuplan_db/nuplan_scenario_filter_utils.py", line 576, in get_scenarios_from_log_file
    this_dict = get_scenarios_from_db_file(parameter)
  File "/workspace/code/autonomous-driving-project/nuplan-devkit/nuplan/planning/scenario_builder/nuplan_db/nuplan_scenario_filter_utils.py", line 165, in get_scenarios_from_db_file
    for row in get_scenarios_from_db(
  File "/workspace/code/autonomous-driving-project/nuplan-devkit/nuplan/database/nuplan_db/nuplan_scenario_queries.py", line 852, in get_scenarios_from_db
    for row in execute_many(query, args, log_file):
  File "/workspace/code/autonomous-driving-project/nuplan-devkit/nuplan/database/nuplan_db/query_session.py", line 21, in execute_many
    cursor.execute(query_text, query_parameters)
sqlite3.DatabaseError: database disk image is malformed

This error is not existing when training on the mini data. I have checked the downloaded blobs, I think they are all complete.

Do you have any idea about how to fix it?

Best wishes

[patk-motional]

Hi @duanyiqun,

This is usually due to your downloaded files being corrupted. Can you try running training on a small subset of the full dataset?