search

zkytony / cos-pomdp

Code for "Towards Optimal Correlational Object Search" | ICRA 2022
Apache License 2.0
13 stars 5 forks source link

Perfect detector (using AI2Thor detector) still using vision detector failure rate #11

Closed RajeshDM closed 4 months ago

RajeshDM commented 4 months ago

I am trying to run the system with a perfect detector but it looks like it might still be using the vision detector probabilities for observation.

https://github.com/zkytony/cos-pomdp/blob/1e3c7fe940a2165b5c5c384ebf047e8139d437ec/cospomdp/models/observation_model.py#L532

Here, the system is checking if the agent is in range - distance wise to see if it can be detected. In cases where it can be detected, it then generates a random number and sees if it's lower than detection probability and then provides the observation based on this. The detection probability here is 0.794 - the detector specs of the vision detector. Shouldn't this be 1 when we are using perfect detector? so the probability of seeing the object if it's in range should be 1 when the object is in range (This is in high level planning so we are not caring about the pitch or yaw)

RajeshDM commented 4 months ago

Hey, any thoughts on this?

zkytony commented 4 months ago

Detector is separate from the sensor model. The detector refers to how the robot receives object detections. Groundtruth detector receives detections by parsing ai2thor's state object. It's still possible to have a groundtruth detector while noisy observation model. I believe that's what I did for COS-POMDP(gt) actually, where the TP/FP rates are still used to configure the observation model (the intention was to ablate on whether yolov5 detector was used and keep everything use the same).

So you may want to figure out how to configure the TP/FP parameters and set TP 100%, FP 0%.

RajeshDM commented 4 months ago

Yea makes sense that the TP has to be set to 100 and FP to 0 for perfect detector.

I now see that detection prob is TP so it definitely makes sense to set that to 100 and and parallely, also need to set FP to 0 while initializing observation model.

Thank you.