search

WHU-USI3DV / FreeReg

[ICLR 2024] FreeReg: Image-to-Point Cloud Registration Leveraging Pretrained Diffusion Models and Monocular Depth Estimators
https://whu-usi3dv.github.io/FreeReg/
219 stars 6 forks source link

On the Calculation of the Final Pose of Registered Images #12

Closed adrianJW421 closed 1 week ago

adrianJW421 commented 2 months ago

Hello!

Based on my understanding, the Tpre matrix generated by the match results is the transformation matrix that converts the extrinsics of the input point cloud (pcd) projection to the camera pose where the image is registered within the point cloud.

Could you provide a example tool code that allows the user to directly obtain the final pose of the registered image? If my understanding is incorrect, could you please explain how to accurately determine the final pose of the registered image?

Thank you!

HpWang-whu commented 1 week ago

Hi @adrianJW421, Many thanks for your suggestion. I have open-sourced a easy-to-use version of FreeReg Here. There are some brief introductions about pose and more easy-to-understand. Welcome to use it!

Yours,

adrianJW421 commented 1 week ago

@HpWang-whu Hello, thanks for releasing FFreeReg, I tried it on the demo data, the registeration looks fine.

After I tried it on some custom image and ply scene, the value of translation vector in the estimated Extrinsic often went to over 100, for example [Result] Estimated intrinsic of image: [[2.99842139e+03 0.00000000e+00 2.04750000e+03] [0.00000000e+00 2.99842139e+03 1.53550000e+03] [0.00000000e+00 0.00000000e+00 1.00000000e+00]] [Result] Estimated extrinsic of image: [[ 9.52254601e-01 -7.50058867e-02 2.95948124e-01 -3.68457958e+02] [ 8.94472803e-02 9.95357155e-01 -3.55431825e-02 1.34429317e+02] [-2.91908135e-01 6.03179138e-02 9.54542503e-01 -4.00320020e+02] [ 0.00000000e+00 0.00000000e+00 0.00000000e+00 1.00000000e+00]]

after visualization, the estimated camera pose and the ply looks like this:

image

Was it purely the performance issue or is there some requirements for preprocessing the input image and ply that I have overlooked? Or is there a particular condition that the ply point cloud should follow (like a certain coordinate system)?

HpWang-whu commented 1 week ago

Hi @adrianJW421 , Thanks for your interest! Can you provide an example raw data to me so I can do some check. The result might be a little late (~ after two weeks) for I am currently going through a very busy month. Really sorry for that!

Yours,

adrianJW421 commented 1 week ago

Hi @adrianJW421 , Thanks for your interest! Can you provide an example raw data to me so I can do some check. The result might be a little late (~ after two weeks) for I am currently going through a very busy month. Really sorry for that!

Yours,

@HpWang-whu Thanks a lot. Can you try download the badcase data from this drive link?

Looking forward to hearing from you

LoveMYChen commented 6 days ago

Excuse me, are you using your own dataset to run the demo