argoverse / argoverse-api

Official GitHub repository for Argoverse dataset
https://www.argoverse.org
Other
830 stars 238 forks source link

Add function to evaluate the stereo competition models #302

Closed jhonykaesemodel closed 2 years ago

jhonykaesemodel commented 2 years ago

This PR adds a function to evaluate the stereo competition models. It computes the number of parametes, number of activations, flops, and latency (inference time) of a given PyTorch model using the fvcore library.

Here is an example of an output (it saves it to a txt file):

Device name: Tesla V100-SXM2-32GB 
Input size 0: torch.Size([1, 3, 10, 10])
Input size 1: torch.Size([1, 3, 10, 10]) 
Mean forward time (ms): 0.20992000202337902 
Std forward time (ms): 0.024825050616623812 
| module               | #parameters or shape   | #flops   | #activations   |
|:---------------------|:-----------------------|:---------|:---------------|
| model                | 1.315K                 | 86.4K    | 2.156K         |
|  conv_layer1         |  0.448K                |  55.296K |  2.048K        |
|   conv_layer1.weight |   (16, 3, 3, 3)        |          |                |
|   conv_layer1.bias   |   (16,)                |          |                |
|  conv_layer2         |  0.867K                |  31.104K |  0.108K        |
|   conv_layer2.weight |   (3, 32, 3, 3)        |          |                |
|   conv_layer2.bias   |   (3,)                 |          |                |

Also, the PR adds the lapsolver to the setup.py. Otherwise test_eval_tracking.py fails during CI checks.