Using vint on own robot, at starting position the closest_node keeps changing even though the robot is at standstill

[vedant-banerjee]

The closest_node at starting point, keeps flickering between 0 and 5 , and when the controller is run, the robot navigates but the closest node detected seems to be wrong , hence the robot is unable to find the goal node and stop, any help on how to debug this issue will be appreciated, even the start and end values are changing even though the robot is at standstill.

[robodhruv]

Hi Vedant,

That is not expected behavior and we're happy to help debug. Would it be possible for you to share a few images from the start, or the entire topomap if possible? What camera are you using?

Dhruv

[vedant-banerjee]

Hi Dhruv , The current update is i have tried fine tuning the model by training it and the issue of finding the closest_node and goal node still seems to be an issue, also my robot is a much larger size compared to the locobot , and there are collisions taking place if obstacles are placed on the path. The camera i am using is Intel Realsense D435I , i have recorded a bag with a small path , which goes only straight and i am trying to use that. I will share the images from the topomap along with my robot image.

[vedant-banerjee]

This is one of the topomaps, but i have tried with a few others as well, and issue of finding closest node and goal node is still there, and collisions are happening. @robodhruv

[robodhruv]

Thanks for sharing, Vedant! I am quite confident that the issue arises from the camera -- RealSense D435i is a fairly narrow FoV (~90 degrees) and quite OOD from what the model is trained on. I would recommend switching to a wide-angle camera, either to replace the D435i, or just add a second USB camera to your build. While this issue is particularly pronounced with ViNT/NoMaD, in general, I expect fisheye cameras to be more useful because the added FoV helps in short-range collision detection and avoidance.

Here is an example camera that you can use: https://www.amazon.com/ELP-Fisheye-Lightburn-Machine-Embedded/dp/B00LQ854AG/

[vedant-banerjee]

Alright thanks for the help, also does the robot size also matter? or this is taken into consideration while fine tuning the model, to avoid collisions @robodhruv

[robodhruv]

The robot size does matter, and can get you in trouble if the robot is too large, but I suspect this will not be a big issue for you unless you are trying to work in a very dense indoor environment. We have tested the model on an AgileX Tracer robot base, which is quite large (50x60 cm) and the model seems to work fine.

[vedant-banerjee]

Okay thanks i will change the camera as suggested and see if i get better results

[robodhruv]

Good luck and feel free to re-open the issue after you've tried!

[vedant-banerjee]

@robodhruv I have tried using a fisheye wide angle lens camera of fov 190 degree, i will share some images of the topomap, i am facing the same issue where the closest_node is not updating correctly as the robot is moving. and hence its losing track. I have not trained/fine tuned it, this is when i run zero shot.