search

lyft / nuscenes-devkit

Devkit for the public 2019 Lyft Level 5 AV Dataset (fork of https://github.com/nutonomy/nuscenes-devkit)
Other
365 stars 103 forks source link

Feature request: Add parallel metric calculation. #53

Open ternaus opened 4 years ago

ternaus commented 4 years ago

At the moment mAP calculation is slow. We can calculate values for different thresholds and classes in parallel.

pyaf commented 4 years ago

Hey @ternaus, I'd love to work on this.

Here's how I plan to implement this:

  1. Parallelly compute the per class mAPs for each IoU threshold and save them in a temp file. (temp files are necessary because parallel processes can't use global variables of the master process, I'm all in for ideas, let me know if you there's a better way to do this.)
  2. Once all parallel processes are finished, read the temp files and calculate and print the overall mAP and remove the temp files.
  3. We can also save all the class-wise computed APs for each threshold in a JSON file, something like this

Let me know your thoughts.