search

yifanlu0227 / HEAL

[ICLR2024] HEAL: An Extensible Framework for Open Heterogeneous Collaborative Perception ➡️ All You Need for Multi-Modality Collaborative Perception!
Other
147 stars 9 forks source link

The metrics calculation #19

Closed CatOneTwo closed 2 months ago

CatOneTwo commented 2 months ago

Could you provide the scripts to calculate the metrics such as FLOPs, Train Throughput, and Peak Memory, Thank you~

yifanlu0227 commented 2 months ago

Hi, you can check here. https://github.com/yifanlu0227/HEAL/tree/main/opencood%2Ftools%2Fprofiler

yifanlu0227 commented 2 months ago

But the usage of those profiler is a little tricky, I preprocess the input data in pickle files. If you want to use these scripts, I can share the pickle files with you.

Or you can develop some new profiler code, just taking mine as a reference.

CatOneTwo commented 2 months ago

But the usage of those profiler is a little tricky, I preprocess the input data in pickle files. If you want to use these scripts, I can share the pickle files with you.

Or you can develop some new profiler code, just taking mine as a reference.

Could you share the preprocess scripts? Thank you~

yifanlu0227 commented 2 months ago

Hi, you can uncomment these code here. https://github.com/yifanlu0227/HEAL/blob/4882bd02514725fa5e2e5717d410de739f33790e/opencood/tools/inference_heter_in_order.py#L191-L202

It will save the agents' input data in pickle format. You may also need to manually select the scene, which has enough agent number that meets your demand. I select a subdataset of range(1220,1260) for OPV2V / OPV2V-H dataset where a agent number of 5 is reached.

https://github.com/yifanlu0227/HEAL/blob/4882bd02514725fa5e2e5717d410de739f33790e/opencood/tools/inference_heter_in_order.py#L170-L171

If you want to profile the single agent's performance, uncomment the code after line 202.

CatOneTwo commented 2 months ago

Thank you~