realsense ros d435 flat surface orientation**

[monian787]

Hi, I am new to realsense and ros,

I was curious to know if the camera can be used to detect the distance and orientation of wall (quite rough wall, need to extract plane first)

let me quickly explain my scenario: it is similar to application of line-following bot wo line

• Robot (~autonomous) needs to stay in the middle of two wall like surface, I have the point cloud data from camera
• i planned to use 2 D435 on either side to to know the distance and orientation of the surface
• with this data i need to use Twist msg to command the wheel velocity

(similar doubt, but closed : https://github.com/IntelRealSense/realsense-ros/issues/1026)

sry if my question is not clear, I will improve with your inputs, thank-you!

[MartyG-RealSense]

Hi @monian787 I carefully considered and researched your need for wall detection and how it might be met.

The first reference that I would suggest is a PDF research paper published recently in 2021 that discusses using D435 and ROS for wall detection on a robot whose application is painting walls.

https://www.mdpi.com/2076-3417/11/4/1467/pdf

Another possibility for navigation, object detection and collision avoidance would be to try Intel's ros_object_analytics package. It is somewhat old now though so may require some adaptation to work with a modern ROS setup.

https://github.com/intel/ros_object_analytics

[monian787]

Thanks for the reply @MartyG-RealSense I'll get back asap

[monian787]

Is there a way to simply extract a plane from the point cloud using ROS.. I looked at lots of literature, but did not understand clearly. Would you recommend any tried out python program (containing ROS subscription to pointcloud2 and plane fitting from there on).. Thanks again

[MartyG-RealSense]

I could not find much in my research regarding obtaining a plane with ROS that was useful enough to share. The guide in the link below looked promising though, especially section 5. RANSAC ground segmentation

https://roboticsknowledgebase.com/wiki/sensing/pcl/

The ROS wall follower project in the link below, which simulates avoidance of a wall either side, may also be a useful reference.

https://github.com/mit-rss/wall_follower_sim

[MartyG-RealSense]

Hi @monian787 Do you require further assistance with this case, please? Thanks!

[monian787]

Hi sorry for the delayed response, But I've been trying few methods.. not arrived at right solution

[MartyG-RealSense]

No problem at all @monian787 - I will keep this case open for a further time period. Good luck!

[monian787]

Hello, thanks for keeping it open,

I am facing an issue with including the libraries, in the following program, pcl_callback, pcl_msg functions are not being recogonised, I've included the pcl library, still there is error,

---------------------Porgram-------------------

!/usr/bin/env python

import libraries and dependancies

import rospy import pyrealsense2 as rs from sensor_msgs.msg import PointCloud2, PointField import sensor_msgs.point_cloud2 as pc2 import numpy import struct from std_msgs.msg import Header import pcl from pylint import epylint as lint

''' main program Subscribing to pointcloud from realsense '''

------------ 1. ROS Node initialization & topic subscription

rospy.init_node('plane_clustering',anonymous=True)

create subscribers

pcl_sub = rospy.Subscriber('/camera/depth/color/points',pc2.PointCloud2, pcl_callback, queue_size=1)

pcl_sub = rospy.Subscriber('/camera/depth/color/points',pc2.PointCloud2, pcl_callback, queue_size=1)

------------ 2. Downsampling - Voxel Grid

vox = plc_msg.make_voxel_grid_filter() LEAF_SIZE = 0.01 # sampling restricted to 1 cm^3

Set the voxel (or leaf) size

vox.set_leaf_size(LEAF_SIZE, LEAF_SIZE, LEAF_SIZE) downsampled = vox.filter()

------------ 3. Region Cropping -- xyz

PassThrough Filter

passthrough = outliers_removed.make_passthrough_filter()

Assign axis and range to the passthrough filter object.

filter_axis = 'z' passthrough.set_filter_field_name(filter_axis) axis_min = 0.2 axis_max = 1.2 passthrough.set_filter_limits(axis_min, axis_max) passed = passthrough.filter()

Limiting on the Y axis too to avoid having the bins recognized as snacks

passthrough = passed.make_passthrough_filter()

Assign axis and range to the passthrough filter object.

filter_axis = 'y' passthrough.set_filter_field_name(filter_axis) axis_min = -0.5 axis_max = +0.5 passthrough.set_filter_limits(axis_min, axis_max) passed = passthrough.filter()

------------ 4. RANSAC segmentation extract the plane

seg = passed.make_segmenter() seg.set_model_type(pcl.SACMODEL_PLANE) seg.set_method_type(pcl.SAC_RANSAC) max_distance = LEAF_SIZE seg.set_distance_threshold(max_distance) inliers, coefficients = seg.segment()

Extract inliers (plane from the bushy enviornment) and outliers

Extract inliers

cloud_table = cloud_filtered.extract(inliers, negative=False)

Extract outliers

cloud_objects = cloud_filtered.extract(inliers, negative=True)

------------------- error message-------------------------------- Traceback (most recent call last): File "./rs_plane.py", line 22, in pcl_sub = rospy.Subscriber('/camera/depth/color/points',pc2.PointCloud2, pcl_callback, queue_size=1) NameError: name 'pcl_callback' is not defined

pl. help, thankyou

[MartyG-RealSense]

In a pyrealsense2 program, I would recommend always having import pyrealsense2 as rs as the first line of the list of imports (i.e above the rospy import instruction).

It looks as though the subscriber code for pcl_callback that you are using is similar to code in the ROS PCL tutorial in the link below.

https://roboticsknowledgebase.com/wiki/sensing/pcl/

Can you confirm please that the node that you are initiating with rospy.init_node('plane_clustering',anonymous=True) does have the name 'plane_clustering'

The guide that I linked to above states that pcl_callback should be called every time the sensor publishes a new pc2.PointCloud2 message.

A step in that guide that is not mentioned in your own procedure is to launch the ROS node before callng rospy.init_node('plane_clustering',anonymous=True). Has roslaunch been performed before you run your pyrealsense2 script please?

roslaunch realsense2_camera rs_camera.launch filters:=pointcloud

If ROS has not been launched then there would not be a pointcloud being published.

[MartyG-RealSense]

Hi @monian787 Do you require further assistance with this case, please? Thanks!

[MartyG-RealSense]

Case closed due to no further comments received.