Question about the estimation process of voxel-wise uncertainty

astra-vision / PaSCo

[CVPR 2024 Oral, Best Paper Award Candidate] Official repository of "PaSCo: Urban 3D Panoptic Scene Completion with Uncertainty Awareness"

https://astra-vision.github.io/PaSCo/

Apache License 2.0

156 stars 15 forks source link

Question about the estimation process of voxel-wise uncertainty #5

Closed Chen-Albert-FENG closed 4 months ago

Chen-Albert-FENG commented 4 months ago

Congratulations! Very impressive work!

However, the manuscript greatly confuses me, especially in uncertainty estimation. I can understand the final PSC output can be ensembled from multiple outputs by averaging the matched semantic and binary mask probability. However, how to obtain the uncertainty of each voxel according to ensemble PSC? Model predicts? How to supervise this uncertainty or unsupervised learning? I cannot get related information from your paper. It's interesting part but I think it's not clear enough to me. I hope the authors can address this question when you feel free.

Thank you for your time!

anhquancao commented 4 months ago

Hi @Chen-Albert-FENG , Thanks for your nice words! You can get the answer from this figure.

Chen-Albert-FENG commented 4 months ago

Thanks for your reply! @anhquancao

Let's firstly focus on the discussion of voxel-wise uncertainty. From your given figure, I suppose the final uncertainty based on softmax operations on 3 SSC probability? For example, for one voxel index, the probability array is [0.5, 0.6, 0.7], how to get the uncertainty? I'm sorry for my understanding ability but I hope you can help me figure out the process when you feel free since this work is very interesting!

Looking forward to your reply!

anhquancao commented 4 months ago

Based on your example, the confidence level is calculated as the average of the values [0.5, 0.6, 0.7], which results in 0.6.

Uncertainty estimation is typically considered as predicting the confidence of the prediction. Higher confidence indicates a higher likelihood of being correct. I recommend reading the following papers for more insights:

Hendrycks et al. "A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks." ICLR 2017.
Lakshminarayanan et al. "Simple and Scalable Predictive Uncertainty Estimation Using Deep Ensembles." NeurIPS 2017.
Havasi et al. "Training Independent Subnetworks for Robust Prediction." ICLR 2021.

Chen-Albert-FENG commented 4 months ago

Thank you for your quick response!