[Autonomy] Vision-based Obstacle Segmentation & Detection

[SwapnilPande]

Which section of robot code is this for? Vision (Obstacle Detection)

Description of feature

• Develop software to detect/segment obstacles (boulders & craters) in the environment
  • Estimate location of obstacle on the field
  • Build map of obstacles for route planning
  • Potential emergency avoidance techniques
• Build ROS node

[partlygloudy]

Things we should look into right away:

1. If we filter out noisy areas of the disparity image, how clearly do rocks show up against a background of sand (i.e. can disparity image alone be used to identify obstacles)
2. Can we use edge detection to supplement the disparity image (maybe by only keeping regions of the disparity image near edges or something along those lines)

Other possible approaches to investigate:

1. Using disparity image (and possibly range information from Kinect), estimate the plane that best fits the ground. Then find locations where the estimated z-coordinate of the ground extends above or below that plane past a certain threshold - these regions are the obstacles. This approach would be difficult to implement but I think it would produce pretty good results. We'd also have to filter out the walls of the competition area somehow.
2. Pray that we don't hit the obstacles

[Dankev55]

We have begun experimentation with the Xbox Kinect (v1, model 1414). Using the Kinect should allow us to gather information on depth images for the purpose of finding obstacles. The goal is to use this data to create a 3D occupancy grid. We are currently considering the OctoMap library for this purpose. The OctoMap uses octrees and probabilistic occupancy estimation. It should give us a representation of occupied, free, and unknown space. This should allow us to recognize and map the locations of obstacles in a 3-dimensional space.

[Dankev55]

With our focus shifting to working with the Kinect and OctoMap, working with image processing from a camera is de-prioritized, and may not be necessary. With regard to this, we made a basic Python script that can analyze a list of similar images and set the grayscale value of any noisy pixel to white, leaving behind only pixels consistent in grayscale value over each image. With the disparity image, such pixels imply a well-defined, non-repeating structure indicative of a protrusion or obstacle. In other words, we should be able to filter out noise and find obstacles using disparity images, if necessary.

[Dankev55]

We are no longer currently pursuing the use of OctoMap, and are looking into the ANYbotics elevation_mapping library. This library will ideally allow us to convert PointCloud2 information from Freenect directly to an elevation mapping visualization that we can use. We have made progress in running a simulated demo using the elevation_mapping library and its associated dependencies, but have an error involving an unknown_published for a 'map' variable.