search

ybkscht / EfficientPose

Other
249 stars 69 forks source link

Inference on LineMOD dataset doesn't work #62

Open saeedEnte opened 1 year ago

saeedEnte commented 1 year ago

Hi,

thank you @ybkscht for the impelementation. It is clear and good documented. The problem that I have: I download a trained network whose link have been provided in ReadMe (I used the object-15 folder). Considering that object-15 means that the network has been trained on the 15th folder of data in processed-LineMOD, I have tried to take an inference from an rgb image from that sub-directory. The model produces NaN for the rotation and translation head. The detection is also not that meaningful (100 boxes have been detected with the confidence of 1.0)

I have tried several objects to be detected. For the camera intrinsic, I suppose that the method in inference.py delivers the right parameters.

Anohter question that I have, the implementation (cost-function, more precisely) takes rotation and translation parameters. What is exactly the dimension of the rotation parameter? In the gt.yml it is a 3*3 matrix but the model produces vector of length 3 for each box. I cannot follow the logic of the dimensions here.

Thanks in advance for your answer.

ybkscht commented 1 year ago

Hi @saeedEnte,

it sounds like something goes wrong during inference because your assumptions seem to be correct. Do you use the inference.py script or are you using some custom inference script?

EfficientPose uses rotations in the axis angle format instead of a rotation matrix. Therefore it produces rotation vectors of length 3 as you mentioned and not 3*3 rotation matrices. The GT rotations are converted from matrix to axis angle vector in the generator when loading the dataset annotations. For example in generators/linemod.py line 433: annotations["rotations"][0, :-2] = self.transform_rotation(np.array(gt["cam_R_m2c"]), self.rotation_representation) which calls the transform_rotation function defined in generators/common.py line 764.

saeedEnte commented 1 year ago

Thank you for the detailed answer. I have used the provided inference.py file and modified the fields for model and input directory.

ybkscht commented 1 year ago

Did you also adjust the class_to_name dictionary in inference.py accordingly?

saeedEnte commented 1 year ago

I have tried the 8th folder with "dirller" (as I have seen that you commented in an issue that 8th object is driller) But still did not work. (With the network trained on 8th Object.) The dictionray should always have the key 0 for just one object (Otherweise I get a KeyError)