Requirements/hypothesis to Validate for Uncertainty estimation

[deebuls]

Writing down the requirements which we dicusses in the class.

This issue will list the differet set of requirements/hypothesis we want to validate about the uncertainty estimation method using our blender datasets

[deebuls]

[Question ] Do Uncertainty estimation methods learn about the object distance/proportion [Hypothesis ] When the object in image is near/high proportion then the uncertainty should be low when it is far then the uncertainty should be high

[deebuls]

[Question ] Do Uncertainty estimation methods learn about the lightning conditions [Hypothesis ] When the object in image is in acceptable luminance(need to define what is acceptable) then the uncertainty should be low when it is not then the uncertainty should be high.

[SathwikPanchangam]

[Question] Impact of background in uncertainty estimation methods. [Hypothesis] If the background is not similar with what was trained the uncertainty should be high.

[deebuls]

[Question] Uncertainty estimation for domain transfer - Trained on synthetic images and tested on real images and vice versa [Hypothesis] The uncertainty should be higher

[SathwikPanchangam]

1. Normal lighting condition image shall have lower uncertainty than abnormal
2. Objects which are far shall have higher uncertainty than normal
3. Shape of the object- deformation
4. Texture
5. Background -- No change in uncertainty
6. Fog -- augmentation
7. Adding similar confusing objects
8. Blurry -- focus
   • camera parameters -- normal parameters
9. Camera angle
10. Occlusion
11. Color of the light source
12. Given trained on a single objects
    • If image has 2 objects then the entropy remains same (uncertainty should not change)