How can we set sched_priority when we use dispatcher_executor

[alexleel]

Issue template

I check rclc_dispatcher_executor_spin implementation , and seems there is no apparent difference in case that I have set the priority.

Steps to reproduce the issue

subscriber code:

‘rclc_executor_sched_parameter_t` sched_p1;
  rclc_executor_sched_parameter_t sched_p2;
  sched_p1.policy = SCHED_FIFO;
  sched_p1.param.sched_priority = 10;
  sched_p2.policy = SCHED_FIFO;
  sched_p2.param.sched_priority = 20;

  rc = rclc_dispatcher_executor_add_subscription(
    &executor, &sub_a, &sub_msg_a, &callback_a,
    ON_NEW_DATA, &sched_p1);
  if (rc != RCL_RET_OK) {
    printf("Error in rclc_dispatcher_executor_add_subscription. \n");
  }

  rc = rclc_dispatcher_executor_add_subscription(
    &executor, &sub_b, &sub_msg_b, &callback_b,
    ON_NEW_DATA, &sched_p2);
  if (rc != RCL_RET_OK) {
    printf("Error in rclc_dispatcher_executor_add_subscription. \n");
  }

  rclc_dispatcher_executor_spin(&executor);

Expected behavior

See the difference of sub_a and sub_b

Actual behavior

No difference since checking executor->handles one by one in rclc_dispatcher_executor_spin()

Additional information

[fujitatomoya]

rclc_dispatcher_executor_spin rclc_dispatcher_executor_add_subscription

where these functions are implemented? can you point out the source file, since i could not find it with https://github.com/ros2/rclc/commit/d263be2057e39f1f702f014548c60f06b5a51333

[JanStaschulat]

Hey @alexleel , to which repository are you referring to. I have started porting the micro-ros repo and have started the implementation in this branch https://github.com/ros2/rclc/tree/feature/restructured-executor/rclc_multi_threaded_executor

But independent of this, how did you test, that there is no difference?

No difference since checking executor->handles one by one in rclc_dispatcher_executor_spin()

Yes that is right. The executor spin-function only dispatches new incoming messages sequentially to multiple threads. The processing of the callback actually happens in DIFFERENT threads with user-defined priority.

So, you could test this, by overloading the system: sending topic A and topic B with high frequency and processing subscription_A in high prio thread and subscription_B in low prio thread. Then you should see that subscription _A is executed more often than subscription_B.

See also this publication: https://arxiv.org/abs/2105.05590

[JanStaschulat]

@fujitatomoya

rclc_dispatcher_executor_spin rclc_dispatcher_executor_add_subscription

where these functions are implemented? can you point out the source file, since i could not find it with d263be2

Probably this development branch: https://github.com/micro-ROS/rclc/tree/feature/restructured-exectutor/rclc_dispatcher_executor

[alexleel]

Hey @alexleel , to which repository are you referring to. I have started porting the micro-ros repo and have started the implementation in this branch https://github.com/ros2/rclc/tree/feature/restructured-executor/rclc_multi_threaded_executor

I checked rclc_dispatcher_executor, it's type-switch multiple thread implementation, rclc_multi_threaded_executor is the better choice? btw, the compilation of this branch failed as following:

Starting >>> rclc_multi_threaded_executor                                        
--- stderr: rclc_multi_threaded_executor                                                                                                     
/home/echo/ros/tmp/rclc/rclc_multi_threaded_executor/src/rclc_multi_threaded_executor/multi_threaded_executor.c: In function ‘rclc_multi_threaded_executor_configure’:
/home/echo/ros/tmp/rclc/rclc_multi_threaded_executor/src/rclc_multi_threaded_executor/multi_threaded_executor.c:29:20: error: ‘rclc_multi_threaded_init’ undeclared (first use in this function); did you mean ‘rclc_multi_threaded_executor_init’?
   29 |   executor->init = rclc_multi_threaded_init;
      |                    ^~~~~~~~~~~~~~~~~~~~~~~~
      |                    rclc_multi_threaded_executor_init
/home/echo/ros/tmp/rclc/rclc_multi_threaded_executor/src/rclc_multi_threaded_executor/multi_threaded_executor.c:29:20: note: each undeclared identifier is reported only once for each function it appears in
/home/echo/ros/tmp/rclc/rclc_multi_threaded_executor/src/rclc_multi_threaded_executor/multi_threaded_executor.c: In function ‘rclc_multi_threaded_executor_handle_init’:
/home/echo/ros/tmp/rclc/rclc_multi_threaded_executor/src/rclc_multi_threaded_executor/multi_threaded_executor.c:65:37: error: lvalue required as left operand of assignment
   65 |   get_multi_threaded_handle(handle) = executor->allocator->allocate(
      |                                     ^
/home/echo/ros/tmp/rclc/rclc_multi_threaded_executor/src/rclc_multi_threaded_executor/multi_threaded_executor.c:67:21: error: ‘rclc_executor_handle_t’ has no member named ‘multi_threaded’
   67 |   if (NULL == handle->multi_threaded) {
      |                     ^~
/home/echo/ros/tmp/rclc/rclc_multi_threaded_executor/src/rclc_multi_threaded_executor/multi_threaded_executor.c: In function ‘rclc_multi_threaded_executor_init’:
/home/echo/ros/tmp/rclc/rclc_multi_threaded_executor/src/rclc_multi_threaded_executor/multi_threaded_executor.c:97:41: error: lvalue required as left operand of assignment
   97 |   get_multi_threaded_executor(executor) = executor->allocator->allocate(
      |                                         ^
/home/echo/ros/tmp/rclc/rclc_multi_threaded_executor/src/rclc_multi_threaded_executor/multi_threaded_executor.c:99:41: warning: passing argument 1 of ‘get_multi_threaded_handle’ from incompatible pointer type [-Wincompatible-pointer-types]
   99 |   if (NULL == get_multi_threaded_handle(executor)) {
      |                                         ^~~~~~~~
      |                                         |
      |                                         rclc_executor_t * {aka struct rclc_executor_t_ *}
/home/echo/ros/tmp/rclc/rclc_multi_threaded_executor/src/rclc_multi_threaded_executor/multi_threaded_executor.c:35:52: note: expected ‘rclc_executor_handle_t *’ but argument is of type ‘rclc_executor_t *’ {aka ‘struct rclc_executor_t_ *’}
   35 | get_multi_threaded_handle(rclc_executor_handle_t * handle)
      |                           ~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~
/home/echo/ros/tmp/rclc/rclc_multi_threaded_executor/src/rclc_multi_threaded_executor/multi_threaded_executor.c: In function ‘rclc_multi_threaded_executor_subscription_set_sched_param’:
/home/echo/ros/tmp/rclc/rclc_multi_threaded_executor/src/rclc_multi_threaded_executor/multi_threaded_executor.c:123:7: error: ‘unsinged’ undeclared (first use in this function)
  123 |   for(unsinged int i = 0; i< executor->max_handles; i++)
      |       ^~~~~~~~
/home/echo/ros/tmp/rclc/rclc_multi_threaded_executor/src/rclc_multi_threaded_executor/multi_threaded_executor.c:123:15: error: expected ‘;’ before ‘int’
  123 |   for(unsinged int i = 0; i< executor->max_handles; i++)
      |               ^~~~
      |               ;
/home/echo/ros/tmp/rclc/rclc_multi_threaded_executor/src/rclc_multi_threaded_executor/multi_threaded_executor.c:123:27: error: ‘i’ undeclared (first use in this function)
  123 |   for(unsinged int i = 0; i< executor->max_handles; i++)
      |                           ^
/home/echo/ros/tmp/rclc/rclc_multi_threaded_executor/src/rclc_multi_threaded_executor/multi_threaded_executor.c:124:25: error: expected expression before ‘[’ token
  124 |   if (executor->handles[[executor->index].type == RCLC_SUBSCRIPTION]) &&
      |                         ^
/home/echo/ros/tmp/rclc/rclc_multi_threaded_executor/src/rclc_multi_threaded_executor/multi_threaded_executor.c:125:3: error: expected identifier before ‘(’ token
  125 |   (executor->handles[executor->index].subscription == subscription)
      |   ^
gmake[2]: *** [CMakeFiles/rclc_multi_threaded_executor.dir/build.make:79: CMakeFiles/rclc_multi_threaded_executor.dir/src/rclc_multi_threaded_executor/multi_threaded_executor.c.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:140: CMakeFiles/rclc_multi_threaded_executor.dir/all] Error 2

@JanStaschulat Can you please help to check it?