sudden change in cuboid angle

[mmahdavian]

Hi shichaoy

Thank You for this great package that you have developed. I have tried to get cuboids from some chairs in my own dataset and sometimes I see sudden changes or even wrong calculations in the results. Is this natural? or there is a problem that I need to fix. I attached a video that you can observe my results.

Thank You

video_2020-11-04_15-44-18.mp4.zip

[shichaoy]

@mmahdavian thanks for trying out, is this per-frame object detection or slam? two main reasons I could think of:

1. The first object is not in full field of view, if bottom of the objects are not seen, it will cause big problem for the detection, because the algorithm assume the bottom line of 2D detected bounding box is the object bottom face. so it will detect bad and wrong posed cuboid.
2. the second case is I seem to also meet before, when the object is facing right towards or vertical to camer (only see one face of an object) in this case, we cannot clear see the three direction lines (cuboid has xyz axis) therefore in the cuboid selection part, it may select the wrong cuboid. I only use geometric way to select cuboid, I think more and more deep learning methods may predict better pose.

[mmahdavian]

Hi @shichaoy Thank you for your reply. I have some questions that I appreciate it if you could answer. 1- The cuboid detections that I have done is from per-frame object detection. I felt like that is what you have done in pred_3d_obj_matched_txt folder files. Can slam make it better and how should I do that? 2 - I have a trained deep learning model that I can use. But I don't know how should I add the angle information to you algorithm, so it would calculate and show cuboids based on my deep learning model. Could you please help me with that? 3 - If I still want to try your method, do you know how should I find out if the whole image can be seen and no part of object is out of image(specially the lower part of object)?

Thank You

[mmahdavian]

Hi @shichaoy

I ran orb_object_slam and I see weird stuff happening in the map creation. At first it doesn't recognize the cuboids that I have given to it by offline detection(first 2 chairs) and then suddenly everything gets messed up. It recognizes them in weird and wrong location with wrong angles. Do you know what might be wrong here? Is that because of wrong cuboid angle detection? I've attached the video for you.

Thank You

simplescreenrecorder-2020-09-03_10.27.40-2020-11-10_10.47.41.mp4.zip

[Liuwenqin-hub]

Hi @shichaoy Thank you for your reply. I have some questions that I appreciate it if you could answer. 1- The cuboid detections that I have done is from per-frame object detection. I felt like that is what you have done in pred_3d_obj_matched_txt folder files. Can slam make it better and how should I do that? 2 - I have a trained deep learning model that I can use. But I don't know how should I add the angle information to you algorithm, so it would calculate and show cuboids based on my deep learning model. Could you please help me with that? 3 - If I still want to try your method, do you know how should I find out if the whole image can be seen and no part of object is out of image(specially the lower part of object)?

Thank You I face the same problem as you when i run this code on my own image dataset. Have you find out the reason to make this problem?