after ransac, frame 229 and 151 has too few matches

[kevinDrawn]

Hello. I'm adapting to using my custom data.

However, while running the 'run_custom.py' with my custom data, numerous matching warnings frequently appear. Consequently, this significantly affects the precision of positioning. I have considered several methods to address this issue:

Increasing the video's fps (from 15FPS to 30FPS). Tuning hyperparameters. Finding the exact intrinsic matrix. Questions:

1. Could you suggest additional tips for improving matching success?
2. I obtained the camera intrinsic matrix using the RealSense SDK function. Should I verify my camera intrinsic matrix in another way? And Color intrinsic matrix is right?

ps. I satisfied all data format like below. Depth: 580.png: PNG image data, 640 x 480, 16-bit grayscale, non-interlaced Mask: 580.png: PNG image data, 640 x 480, 8-bit grayscale, non-interlaced RGB: 580.png: PNG image data, 640 x 480, 8-bit/color RGB, non-interlaced

I'm sure there is no problem in depth camera setting. This is my cloud_init.ply

So, I make longer the distance between camera and objects. Because my camera D415 depth range is 0.5m~3m.

Therefore, the initial cloud.ply is better.

But, I don't know why the shadow of the object is appeared.

[wenbowen123]

It seems the wall of the box is curved. Can you share the ply file?

[Eric-nguyen1402]

@kevinDrawn Hi Can you please tell me how to create a custom data step by step and how long does it take to execute the video like instruction? Because I tried to use my custom data but it did not run well. Thank you so much

[kevinDrawn]

Sorry for the late reply, @wenbowen123! I've solved my problem. The depth range of my camera is over 0.5 meters, but I filmed my custom data from too close a distance. As a result, the point cloud data disappeared, and RANSAC mismatching occurred.

[kevinDrawn]

@Eric-nguyen1402

1. Prepare rgb, depth data (Depth data type uint16!!, and use rs.align to match the depth image and depth image.
2. Get Annotation mask data of first frame and feed to the XMem. You can find how to use the custom data by XMem.
   • you shoud locate first frame's mask image named 'mask.png' in the same directory 'rgb', 'depth','masks',
   • All of the mask images locate to masks folder.
3. And Run

[Eric-nguyen1402]

@kevinDrawn I got it. Thank you so much. Btw, have u tried to run it online with XMem?

[kevinDrawn]

@Eric-nguyen1402 Not yet. I didn't make pipeline including XMem.

[Eric-nguyen1402]

@kevinDrawn Ok. Thanks. I am trying to do it now