ApexAI / performance_test

**This project is deprecated** Go to https://gitlab.com/ApexAI/performance_test
64 stars 41 forks source link

Print RMW implementation in log #97

Closed pbaughman closed 5 years ago

pbaughman commented 5 years ago

This calls the rmw.h function rmw_get_implementation_identifier so the resulting graph of the test results can include the RMW implementation.

log_Array1k_11-10-2019_10-58-42.pdf

In the attached report, notice that the information is included in the environment section in the lower-right

Signed-off-by: Pete Baughman pete.baughman@apex.ai


This change is Reviewable

daggarwa commented 5 years ago

@pbaughman I am noticing that the the RMW Implementation is getting printed two times at the start of the experiment on the console :

divya.aggarwal@ade:~/perf_test_ws$ ros2 run performance_test perf_test -c ROS2 -l log -t PointCloud4m --history_depth 100 --max_runtime 60 -p 1 -s 1 -r 20 --reliable --keep_last
RMW Implementation: rmw_apex_dds
Experiment id: d3746c80-b618-49b4-9d4b-15391dbce0db
perf_test version: 34a66d9
Logfile name: log_PointCloud4m_14-10-2019_09-37-53
Communication mean: ROS2
RMW Implementation: rmw_apex_dds
DDS domain id: 0
QOS: Reliability: RELIABLE Durability: VOLATILE History kind: KEEP_LAST History depth: 100 Sync. pub/sub: 0
Publishing rate: 20
Topic name: PointCloud4m
Maximum runtime (sec): 60
Number of publishers: 1
Number of subscribers:1
Memory check enabled: 0
Use ros SHM: 0
Use single participant: 0
Not using waitset: 0
Not using Connext DDS Micro INTRA: 0
With security: 0
Roundtrip Mode: NONE
---EXPERIMENT-START---
T_experiment,   T_loop, received,   sent,   lost,   relative_loss,  data_received,  latency_min (ms),   latency_max (ms),   latency_mean (ms),  latency_variance (ms),  pub_loop_res_min (ms),  pub_loop_res_max (ms),  pub_loop_res_mean (ms), pub_loop_res_variance (ms), sub_loop_res_min (ms),  sub_loop_res_max (ms),  sub_loop_res_mean (ms), sub_loop_res_variance (ms), ru_utime,   ru_stime,   ru_maxrss,  ru_ixrss,   ru_idrss,   ru_isrss,ru_minflt, ru_majflt,  ru_nswap,   ru_inblock, ru_oublock, ru_msgsnd,  ru_msgrcv,  ru_nsignals,    ru_nvcsw,   ru_nivcsw,  
1.038968,       1.038965,       6,      3,      0,      0.00,       28258545,7.028,     1709,       253.3,      353.3,      -1656,      44.59,      -239.6,     401.5,  -725.1,     49.98,      -31.96,     44.21,      1.124,      1.703,      7283964,        0,      0,      0,      1817106,        0,      0,      0,      8,      0,      0,      0,      160,        9,      
2.089596,       1.050510,       19,     20,     0,      0.00,       83845178,8.771,     13.1,       11.79,      0.0009947,      42.7,       44.42,      42.96,      0.000119,       -1.434,     49.97,      24.99,      0.6238,     1.424,      1.706,      7283964,        0,      0,      0,      1817107,        0,      0,      0,      8,      0,      0,      0,      312,        10,     
pbaughman commented 5 years ago

@daggarwa The first one is being printed to stderr by your DDS implementation - performance_test has no control over that.

If you build 'master' you will still see the first one. If you direct stdout to /dev/null, you will still see the first message, because it's coming out over stderr

ros2 run performance_test perf_test -c ROS2 -l log -t Array1k --max_runtime 10 > /dev/null
daggarwa commented 5 years ago

@pbaughman I have refactored the code to keep up with the existing design. Rest everything looks good. Will merge the PR after CI passes

pbaughman commented 5 years ago

@daggarwa I believe your modifications to this PR introduced a breaking change. If I build 875be578e2cbc56d8ec5473daf501441417d6736, I can run

ros2 run performance_test perf_test --help

Successfully.

Starting with ce1b21ac2218ff558e0f9f6c214ec56c0f234122 when I run perf_test --help, I get

Version: ERROR: Experiment is not yet setup!

. . printed before the help message, and also an exit code of '1' .

The error message when running with --help is sub-optimal, but the exit code of 1 is problematic. In our CI jobs, we run perf_test --help as a sanity check and that sanity check is now failing.

Specifically, I believe the check_setup calls in the below snippit are unnecessary:

std::string ExperimentConfiguration::rmw_implementation() const
{
  check_setup();
  return m_rmw_implementation;
}

 std::string ExperimentConfiguration::perf_test_version() const
{
  check_setup();
  return m_perf_test_version;
}

While the RMW implementation might (I honestly don't know) depend on some other argument passed to performance_test, the githash of the source code used to build performance_test surely does not.