search

spencer-project / spencer_people_tracking

Multi-modal ROS-based people detection and tracking framework for mobile robots developed within the context of the EU FP7 project SPENCER.
http://www.spencer.eu/
660 stars 327 forks source link

Run example from a single .bagfile #51

Closed iqedgarmg closed 5 years ago

iqedgarmg commented 6 years ago

Hi everyone!

I am trying to run the first tutorial using a single .bag file (in order to do this with my own RGB-D camera). For this purpose I did the following procedure:

1.- Merge the .bag files I merge 2 of the .bag files that are generated with the download_example_bagfiles.sh script (front_rgbd.bag and tf_odom.bag). This new .bag file called "Spencer.bag" has the following topics:

2.- Modify launch file I uncomment and modify the following lines on tracking_on_bagfile.launch file:

But altough the RGB point cloud is showed on Rviz, no detection is processed. If I try with separate .bag files (front_rgbd.bag and tf_odom.bag) on my own folder using the "folder" arg the node works fine.

I really appreciate any help to solve this issue.

Thanks in advance for your response

tlind commented 5 years ago

Not sure why you are merging the bagfiles, as rosbag play can automatically do that during playback. But nevertheless, it should work.

I would suggest to use rosnode list and rosnode info to go through the nodes (starting at the bottom with the srl_nearest_neighbor_tracker or nnt_node), to see if required topics are unconnected.

kivrakh commented 5 years ago

Hello,

I had the same issue with @iqedgarmg. To try the framework with my own bag data:

First of all, I carefully investigated the rosnode relationships and came up with the same topics as @iqedgarmg and recorded:

Then use "image_transport" package to uncompress or republish:

Finally use "openni2_launch" to publish:

Although I see all the connections are done on rqt_graph and echo the topic. I cannot see any detection. This issue took me two days. I will be grateful if you could help us!

Thank you in advance.

iqedgarmg commented 5 years ago

Hi @kivrakh, I remember that the problem was related to the stream codec of the depth, that needs to be the same that the example bag. You can do this with an OpenCV function, and check the correct codec with the "encoding" header of the rosbag example provided by the package..

You can check this and if you need help I can recheck in my old files.

Greetings!

kivrakh commented 5 years ago

Hi @iqedgarmg, I would be happy if you could check your solution and share with all of us. I think everyone who creates their own bag files encounters a similar problem!. I really appreciate that!

kivrakh commented 5 years ago

When I run the following command roslaunch spencer_people_tracking_launch tracking_on_bagfile.launch

The encodings of: rostopic echo /spencer/sensors/rgbd_front_top/rgb/image_rect_color/encoding rostopic echo /spencer/sensors/rgbd_front_top/depth/image_rect/encoding

are 32FC1 and rgb8, respectively. They are the same with my bag file outputs topic encodings..

kivrakh commented 5 years ago

Finally, I got the point. The problem was related to timestamps synchronization. Because upper_body_detector uses message time synchronizer applied to the depth, camera_info and ground plane topics:

That is, all these topics publish rate has to be the same. Ground_plane node rate was 30.0 by default. However, my publish rate of depth image and camera info nodes was about 4Hz. When I synchronize all these topics the problem was solved.

germal commented 3 years ago

Hi @kivrakh , I am trying to adapt Spencer to d435 too.I have my topics syncronized with ground plane but the upper_body_detector still doesn't detect. I modified topics name is modified as well. Could you advice what should I pay attention further ? Thanks & Regards germal

AlessandroMelino commented 3 years ago

Hello @kivrakh and @germal , I am facing similar issues with a Realsense D435 and Upper Body Detector. Did you make it work?

I would ask 2 question:

  1. How can I check if the topics are syncronized?
  2. How is the correct way to set the ground plane?

Thanks in advance. Alessandro

kivrakh commented 3 years ago

Hello @AlessandroMelino, Yes, we did make it work finally.

  1. You should check the publish rate of that three topics by:
  1. In spencer_people_tracking_launch)/launch/detectors/front_rgbd_detectors.launch launch file you can set the publish rate of ground plane using
    • <param name="rate" value="30.0"/>

Hope this helps.

AlessandroMelino commented 3 years ago

Hello @kivrakh, thank you for your answer.

I have checked the camera_info and ground_plane frequencies and they are ok, but the image_rect topic that exists but doesn't publish new messages. Instead I have a topic called /spencer/sensors/rgbd_front_top/depth/image_rect_raw/ which is the right topic the detector should subscribe. I tried remmapping it but didn't work (<remap from="/spencer/sensors/rgbd_front_top/depth/image_rect_raw" to="/spencer/sensors/rgbd_front_top/depth/image_rect"/>).

Am I doing it right?

EDIT: I have achieved to receive the image for the upper body detector, modifying the file front_rgbd_detector.launch like this:

        <!-- Front RGB-D upper body -->
        <include file="$(find rwth_upper_body_detector)/launch/upper_body_detector_realsense.launch" ns="rgbd_front_top" if="$(arg upper_body)">
            <arg name="detected_persons" value="/spencer/perception_internal/detected_persons/rgbd_front_top/upper_body"/>
            <arg name="camera_namespace" value="/spencer/sensors/rgbd_front_top"/>
            <arg name="depth_image" default="/depth/image_rect_raw" />
            <arg name="rgb_image" default="/color/image_raw" />
            <arg name="camera_info_depth" default="/depth/camera_info" />
            <arg name="upper_body_detections" value="upper_body_detector/detections" />
            <arg name="upper_body_bb_centres" value="upper_body_detector/bounding_box_centres" />
            <arg name="upper_body_image" value="image" />
            <arg name="ground_plane" value="$(arg ground_plane)" />
            <arg name="detection_id_offset" value="3"/>
            <arg name="detection_id_increment" value="$(arg detection_id_increment)"/>
        </include>

Nevertheless, I am not still receiving any detection.

Best regards. Alessandro

germal commented 3 years ago

Hello @AlessandroMelino , I've just seen your message, I don't use Spencer since some months but I remember I modified somewhere this frequency.I will getting back tomorrow after checking !

AlessandroMelino commented 3 years ago

Hello @AlessandroMelino , I've just seen your message, I don't use Spencer since some months but I remember I modified somewhere this frequency.I will getting back tomorrow after checking !

Okey, thank you for your answer. I am getting this frequencies if it is useful for you: image

I am getting too this warning when I launch the detector: [ WARN] [1615794471.342697164]: Failed transform lookup in rwth_ground_plane/tf_based_fixed_gp -- maybe the RGB-D drivers are not yet running!? Reason: "rgbd_front_top_depth_optical_frame" passed to lookupTransform argument target_frame does not exist. . Message will re-appear within 20 seconds.

I don't know if this is important, but it doesn't appear any more.

Thanks in advance. Alessandro

germal commented 3 years ago

Hello Alessandro,

I couldn't get access to my code till now , but in the meantime I have found the following notes

modify publishrate 30 -> 15 in the following files

-utils/rwth_ground_plane/src/tf_based_fixed_gp.cpp: private_nodehandle.param("rate", publishRate, 30.0); -utils/rwth_ground_plane/src/really_fixed_gp.cpp: private_nodehandle.param("rate", publishRate, 30.0);

and recompile with catkin_make or catkin build

tweak also this parameter height_above_ground in roslaunch spencer_people_tracking_launch tracking_single_rgbd_sensor.launch

Hope this help in the meantime ! Feel free to contact me on mail Regards

AlessandroMelino commented 3 years ago

Thank you for your answer @germal.

So now I have done these changes and the publish rate of the ground_plane is 15 Hz:

image

But still, no detections.

I have set the height_above_ground parameter in 0.96 meters, as the height of the sensor respect to the ground. Must I try some other values?

By the way, I also set the tf of my sensor in that value. Can this cause any problem?

image

EDIT: what about the config file of the camera located in spencer_people_tracking/detection/rgbd_detectors/rwth_upper_body_detector/config/? Have you modified something?

My depth stream is 640x480.

Thanks in advance for all the help.

Best regards. Alessandro

germal commented 3 years ago

Hey Alessandro, In case you haven't done yet, you have to set depth scale to 1000 in the asus.yaml file.maybe this can resolve. Regards !

kivrakh commented 3 years ago

I advise you to check the depth and rgb image frequency. There is a line 330 of code in ~/catkin_ws/src/spencer_people_tracking-melodic/detection/rgbd_detectors/rwth_upper_body_detector/src/main.cpp. The difference in frequencies should not be much than 0.2.

// Creating a ros image with the detection results an publishing it
if(vis && color_image && abs((depth->header.stamp - color_image->header.stamp).toSec()) < 0.2) {  // only if color image is not outdated (not using a synchronizer!)
AlessandroMelino commented 3 years ago

Hello @germal and @kivrakh.

Finally it worked. It was the depth scale of the .yaml file, I have changed it to 1000 and it is detecting people.

Thanks a lot for all the help.

Best regards. Alessandro

germal commented 3 years ago

Hey Alessandro , I''m glad to hear that I could help you! Regards

kivrakh commented 3 years ago

I am happy that you find a solution to your problem. I have not experienced the depth scale parameter. Thank you all.