search

wayveai / fiery

PyTorch code for the paper "FIERY: Future Instance Segmentation in Bird's-Eye view from Surround Monocular Cameras"
https://wayve.ai/blog/fiery-future-instance-prediction-birds-eye-view
MIT License
549 stars 87 forks source link

clarification evaluation #31

Closed F-Barto closed 2 years ago

F-Barto commented 2 years ago

Hello and many thanks for your work and sharing your code.

I have a question regarding the way you compute your IoU metric and how it compares against Lift-splat.

You use stat_scores_multiple_classes from PLmetrics to compute the iou. Correct me if I am wrong, but by default the threshold of this method is 0.5

On the other hand, in get_batch_iou of LFS they use a threshold of 0: pred = (preds > 0) https://github.com/nv-tlabs/lift-splat-shoot/blob/master/src/tools.py

Wouldn't this have an impact on the evaluation results ,and thus, on how you compare to them ?

F-Barto commented 2 years ago

My bad, LFS uses preds > 0 where preds is before sigmoid, which makes it > 0.5 after sigmoid