waymo-research / waymo-open-dataset

Waymo Open Dataset
https://www.waymo.com/open
Other
2.71k stars 611 forks source link

WOSAC eval locally becomes very slow #877

Open pengzhenghao opened 4 days ago

pengzhenghao commented 4 days ago

Hi,

I notice that the WOSAC eval code becomes very slow after the update to new version of waymo-dataset.

My version:

waymo-open-dataset-tf-2-12-0 1.6.4

The profile information:

image

My code entering waymo_dataset's functions:

        config = load_metrics_config(use_2024=use_2024)
        scenario_metrics = metrics.compute_scenario_metrics_for_bundle(config, scenario, scenario_rollouts)
        aggregate_metrics = metrics.aggregate_metrics_to_buckets(config, scenario_metrics)
pengzhenghao commented 2 days ago

I should mention that it takes 32s on my PC to eval a single scenario (with 32 modes).