dymczykm
commented
6 years ago @PerrineAguiar thanks for your question.
We have experimented with localizing between different types of devices and that indeed causes issues. So the result is somewhat expected. I can point you to few things that are worth checking:
- rotation invariance - make sure the rotation invariance for the descriptors is switched on, otherwise the (possibly) different orientation of the cameras will prevent the descriptors from matching,
- scale and resolution issues - if your cameras significantly differ in the sensor resolution you will end up with descriptor patterns of the same scale (expressed in pixels) covering completely different scale of image patches. Make sure to adapt this.
- lens distortion - as the descriptors are extracted using the distorted image, the significantly differing distortion models of the two lenses might break the matching. One idea would be to operate on the undistorted image (expensive in terms of computation) or adapt the pattern using the distortion model (quite cheap, but you would need to modify the brisk code we're using).
Hope that helps.
PerrineAguiar
Hi,
I built a map using one device and tried to localize in this map using the same device. It worked great! However when using another device to localize, it fails, i.e. no successful localizations, so only VIO is used. Both devices output grayscale fisheye images but lenses are different. Both devices have their intrinsics and IMU-camera extrinsic calibrated. Is this an expected result? Or do you support this use case (different devices for mapping and localizing)?
Many thanks in advance!