a1_perception_node not reproducable

[sacchinbhg]

Hey, so I had the following doubts after going through the benchmark repo thoroughly.

1) The steps given here are not reproducible. A github issue of the same exists and points to the solution here. Following the solution we get this output:

The documentation does not explain what to do after this. There is a /input node which I am not sure if the rosbag should attach too or something else. Is there a ros bag that you guys have?

2) If I do ro2 node info /image_input_component this is what I get

I do not understand why the node is subscribing to the topic "/camera/camera_info" and publishing to "/camera/camera_info".

3) If I publish a rosbag to send data to "/camera/camera_info" and "/camera/image_raw". This happens: image.png

This is due to the image_input_component node publishing and subscribing to the same camera_info. This leads to more camera_info messages coming than camera_rawImages. If you guys can clarify why this happens it would help a lot. Anyhow even with this working it does not seem like I am getting the result as shown in the github.

If you can let me know what the next steps are from here, it would help a lot. Thank you for the same

[mcres]

Perhaps the steps described for a3 benchmark are more detailed: https://github.com/robotperf/benchmarks/blob/main/benchmarks/perception/a3_stereo_image_proc/benchmark.yaml. For the moment, you can easily extrapolate the steps needed for your particular case.

Can you please confirm if you are getting traced data in ~/.ros/tracing/a1_perception_2nodes using babeltrace2 as explained in the link above?

If so, you should be able to run ros2 launch a1_perception_2nodes analyze_a1_perception_2nodes.launch.py and get some latency metrics and plots of your node.

[vmayoral]

The steps given here are not reproducible. A github https://github.com/robotperf/benchmarks/issues/1 of the same exists and points to the solution here. Following the solution we get this output:

You're pointing to the TEMPLATE, of course that's not expected to be reproducible.

I'd recommend reviewing the comments from others and/or to join the ROS 2 HAWG meetings before making statements like this. There's quite a few people working on it daily, and confirming it is reproducible. Moreover this is running in an automated manner daily.

[sacchinbhg]

Hey, so I now tried the a3 node. The steps there were quiet easy to understand. Sorry for that.

Everything works fine until this step: babeltrace a3_stereo_image_proc | less This is the output for that command. Which I think means the trace launch file works well and good.

But when I try ros2 launch a3_stereo_image_proc analyze_a3_stereo_image_proc.launch.py I get this output: Also if I do ros2 node list I get this: Why are there 2 nodes with the same name? Am I doing something wrong?

I will join the next weekly meeting for sure. If there is anything else I can do till then it would really help me out.

[vmayoral]

Closing this ticket. a1's confirmed and fully reproducible across compute platforms. See https://news.accelerationrobotics.com/robotperf-alpha/ for some public results with data digested.

See https://github.com/robotperf/benchmarks/releases/tag/alpha for the complete alpha release.