search

tusen-ai / SST

Code for a series of work in LiDAR perception, including SST (CVPR 22), FSD (NeurIPS 22), FSD++ (TPAMI 23), FSDv2, and CTRL (ICCV 23, oral).
Apache License 2.0
765 stars 96 forks source link

Estimation of FSDv2 inference speed without re-training #184

Closed ArseniuML closed 4 months ago

ArseniuML commented 4 months ago

I've merged lightweight segmentor backbone into fsdv2_argo_2x config and want to estimate speedup. I want to do this without downloading Argoverse2 dataset and re-training FSDv2 - my manager wants to make sure that the time spent downloading the data and re-training FSDv2 will pay off. I've created a random checkpoint, but it seems that it's not helpful - FSDv2 inference time with fsdv2_argo_2x config and your pre-trained checkpoint is 100 ms, and with the same config and random checkpoint is 700 ms. With fsdv2_argo_2x_light config and random checkpoint inference time is also about 700 ms...

Is there a way to estimate FSDv2 inference speed without re-training?

ArseniuML commented 4 months ago

I've tested segmentor backbone with light config and random checkpoint, and there is >3x speedup (15 ms vs 50). But the whole FSDv2 test with a random checkpoint seems not to be representative... If testing with random checkpoint is useless, maybe there is an express method to fast re-train FSDv2 on small part of Argo2?

ArseniuML commented 4 months ago

Downloaded Argoverse2.