search

cheind / py-motmetrics

:bar_chart: Benchmark multiple object trackers (MOT) in Python
MIT License
1.37k stars 258 forks source link

Use dataframe.groupby instead of iterating #61

Closed jvlmdr closed 4 years ago

jvlmdr commented 4 years ago

I noticed that iteratively selecting rows from the dataframe was a serious bottleneck.

It looks like someone was already investigating this. I have removed the use of the cached analysis and the lines which computed timings.

I isolated the code for extracting counts and added a benchmark (and a dependency on pytest-benchmark).

Before:

--------------------------------------------------------- benchmark: 1 tests ---------------------------------------------------------
Name (time in s)                                  Min      Max     Mean  StdDev   Median     IQR  Outliers     OPS  Rounds  Iterations
--------------------------------------------------------------------------------------------------------------------------------------
test_benchmark_extract_counts_from_df_map     15.4156  16.1166  15.6762  0.3507  15.4331  0.6114       1;0  0.0638       5           1
--------------------------------------------------------------------------------------------------------------------------------------

After (time in ms not s):

------------------------------------------------------------ benchmark: 1 tests ------------------------------------------------------------
Name (time in ms)                                  Min       Max      Mean   StdDev    Median      IQR  Outliers     OPS  Rounds  Iterations
--------------------------------------------------------------------------------------------------------------------------------------------
test_benchmark_extract_counts_from_df_map     146.5993  209.5080  175.8946  22.3510  174.9131  17.7610       2;0  5.6852       5           1
--------------------------------------------------------------------------------------------------------------------------------------------
jvlmdr commented 4 years ago

Sorry about the large number of commits. When the pull request was merged, the commits were flattened. I only pulled this collapsed-commit after doing the work. If you like, I can try to rebase it?

jvlmdr commented 4 years ago

Opening new pull request with rebased branch

cheind commented 4 years ago

Very cool, Jack! You are driving this project :) I guess we should think about making you a maintainer, since the time I can spend on professionally on this project has become quite limited. Interested?

jvlmdr commented 4 years ago

Thanks! Yep, I think I would be able to do that. Let's talk via email.