Closed methylDragon closed 2 years ago
If I understood correctly, this adaptive component is a way to expose an interface to swap between node components dynamically.
Correct.
What is the intended interaction with the parameter system for the implemented nodes for the different architectures? Are parameters meant to be bound to the adaptive component (essentially the node manager), or is the intention to have the parameters bound to the architecture specific nodes instead?
The former would make it Component
-agnostic, which is what's seeked in here originally. Otherwise, every single new Component
would need to implement the parameter logic, which IMHO defeats the purpose of this abstraction. But I'm open for discussion. Would the latter bring more value in your view?
adaptive_component
provides by default a composable stateless[^1] container for adaptive ROS 2 Node computations. Essentially, it allows to hot-swap between Components targeting different compute architectures (e.g. FPGA, CPU or GPU) at run-time and without having to stop Nodes. In more elaborated use cases, adaptive_component
should allow you to build stateful adaptive ROS 2 Node computations[^2].
[^1]: Example of stateless adaptive ROS 2 Node https://github.com/ros-acceleration/acceleration_examples/blob/main/nodes/doublevadd_publisher/src/doublevadd_component_adaptive.cpp [^2]: Example of stateful adaptive ROS 2 Node https://github.com/ros-acceleration/acceleration_examples/blob/main/nodes/doublevadd_publisher/src/doublevadd_component_adaptive_stateful.cpp
I think the former makes sense also, but there will need to be an implementation to allow for the sharing of parameters.
In ROS 2, parameters are associated with each node, which means that some extra code interfaces must be implemented in the adaptive_component and/or child components to allow child nodes to access the parameters in the adaptive component.
One way to do this is via a pattern like the parameter_blackboard (src), or by having the child nodes call the services on the adaptive component at init or param change, or potentially using one of the parameter synchronization clients (here)
No matter how you do it, I think this functionality is quite core to the usage of this package.
I think the former makes sense also, but there will need to be an implementation to allow for the sharing of parameters.
In ROS 2, parameters are associated with each node, which means that some extra code interfaces must be implemented in the adaptive_component and/or child components to allow child nodes to access the parameters in the adaptive component.
One way to do this is via a pattern like the parameter_blackboard (src), or by having the child nodes call the services on the adaptive component at init or param change.
No matter how you do it, I think this functionality is quite core to the usage of this package.
Not really (or I just don't follow you). On what basis do you believe that further parameter exchange needs to happen between the Components within an adaptive_component
container? Do you have any toy example that demonstrates this limitation? Have you tested things against the toy example that I provided in here? If you have difficulties testing the FPGA piece, simply replace that Component with another CPU one (that slightly changes the behavior for visual inspection) that allows you to debug this out in your workstation.
The current approach worked just fine in my testing. Note that as highlighted above, adaptive_component
implements a composable stateless container for adaptive ROS computations. It does so by leveraging a single threaded executor (yes, within the Component), which is then used to hold one of the sub-components at a time. Accordingly, the only parameters you're modifying are the ones from the adaptive_component
Component itself, which is the one that takes care of abstracting away the complexity from all sub-components.
Note that the adaptive_component
can still be composed with other Components into a Node, as usual, with any of the executors available in ROS 2.
Suppose we made an adaptive_component
with the following two NodeCPU nodes:
// doublevadd_publisher_adaptive.cpp
...
auto adaptive_node = std::make_shared<composition::AdaptiveComponent>(
"doublevadd_publisher_adaptive", // name of the adaptive Node
options, // Node options
// CPU
std::make_shared<NodeCPU>("_doublevadd_publisher_adaptive_cpu1", options),
// FPGA
std::make_shared<NodeCPU>("_doublevadd_publisher_adaptive_cpu2", options),
// GPU
nullptr);
...
And on init of each component we initialized a random ID and stored it in a parameter
void
DoubleVaddComponent::initialize()
{
// Create a publisher of "std_mgs/String" messages on the "chatter" topic.
pub_ = create_publisher<std_msgs::msg::String>("vector", 10);
this->declare_parameter("example_param");
this->set_parameter(rclcpp::Parameter("example_param", rand()));
RCLCPP_INFO(this->get_logger(), "NODE COMPONENT INIT: ID: %ld", this->get_parameter("example_param").as_int());
// define differential time value (in ms) for the loop
dt_ = 100ms; // 10 Hz
// Use a timer to schedule periodic message publishing.
timer_ = create_wall_timer(dt_, std::bind(&DoubleVaddComponent::on_timer, this));
}
Then, on the component's timer, if we print the random ID... (output below)
RCLCPP_INFO(this->get_logger(), "Publishing: '%s' | Param: %d", message.data.c_str(), this->get_parameter("example_param").as_int());
You'll notice that any set parameters are local to the components. In the output below, I'm swapping between the adaptive nodes.
Reminder: I am only setting the parameter on node component init! They're not set again anywhere else, and the setting always occurs with the printout. Notice how, when we swap components, the init printout doesn't print again, which means that no parameters were set, yet the param value still changes!
(Pay attention to the Param value (which prints the random ID) at the end of each print line.)
Furthermore, those internal components' parameters are not visible in the CLI
Edit: This is just because the names are prepended with _
, which I think was intentional and not an issue, mb
$ ros2 param list
/doublevadd_publisher_adaptive:
adaptive
qos_overrides./parameter_events.publisher.depth
qos_overrides./parameter_events.publisher.durability
qos_overrides./parameter_events.publisher.history
qos_overrides./parameter_events.publisher.reliability
use_sim_time
I think the main thing to be clear on is that the AdaptiveComponent
and its child components are each nodes in their own right with their own parameter namespaces. While it's true that the child nodes are "subsumed" under the AdaptiveComponent, they do not share their parameter namespaces.
All that AdaptiveComponent
is doing is just storing the node pointers and changing which one is used with the executor, but there's no synchronization or mechanism like that that is working with parameters.
--
Aside:
Now, what I am not sure about is why the child node components don't show up in ros2 node list
or ros2 param list
. They by right should be appearing if they're on the executor. And I've confirmed that they can subscribe...
This is a separate issue that might need to be tracked down. If you happen to have insight on this let me know!
Edit: It's just because the node names are prepended with _
, so they're hidden. (No issues here, I just didn't notice, oops.)
Now, we have two problems:
Consider the following use case, company X is integrating their navigation stack, and wants to hotswap between FPGA and CPU computation. They explicitly want their navigation parameters to be updatable, and expect the parameters they set to "stick" regardless of what underlying hardware they're running ROS on.
Under the current implementation:
adaptive: 1
, running B_FPGA, expecting the changes made when they were running A_CPU to also reflect in B_FPGAThere are some approaches to take but all require that some synchronization implementation be folded into the use scenario of this package.
AdaptiveComponent
this->declare_parameter
and associated methods to intercept the call and defer it to the overarching AdaptiveComponent
. (This would probably require one more class interface to do the interception.)
AdaptiveComponent
and instruct users to use that in any downstream node components to get and set parameters instead of using this->
.doublevadd_component_adaptive_stateful.cpp
example, except as part of this package, since I think it's a core enough functionality...I'm still undecided as to which is the best approach though..
This is also made a little bit more complicated in that inactive nodes that are not running on the executor won't be able to subscribe/listen for service callbacks (by design.)
The difficulty in synced siblings like that make me want to just use something in the parent node to synchronize.
Ok, thanks for the rationale above, that helped align visions.
Shortly, I think what you're describing above is a subset of the stateful possibilities (in your example, the state is a ROS parameter you want to pass from Component in CPU1, to Component in CPU2), but adaptive_component
is purposely stateless by default[^1] and can be made stateful, if desired (as exemplified in here, refer to the README for more), but we believe that implementing the state should be up to the user (if needed).
Also, kindly note my previous comment in here and the rationale at [^1]:
Note that as highlighted above, adaptive_component implements a composable stateless container for adaptive ROS computations.
Regarding your comment about:
All that AdaptiveComponent is doing is just storing the node pointers and changing which one is used with the executor, but there's no synchronization or mechanism like that that is working with parameters.
Correct, but that's all we want at this point, a container that facilitates this feature and simplifies the process of building adaptive ROS computations. I agree that stateful adaptive Components are needed, and that's why we exemplified it, but out of all our internal discussions we concluded that that state is actually application-specific (or Component-specific, if you're picky). So it's to the user to specify which mechanisms shall be used and what information should be conveyed across Components in an adaptive_component
container.
Part of this rationale was discussed at https://discourse.ros.org/t/adaptive-ros-2-node-computations-and-hardware-acceleration-contributions/23197. Let me know if this addresses your concerns above, and also, please send any further feedback that comes to mind.
[^1]: Because simply you can't contemplate all possibilities of what a state will represent in a ROS Component. In some cases, as in our example, it'll be just a protected variable in a subclass of AdaptiveComponent
. In others, for example in yours, the state is a ROS parameter which you initialize in the source code of one of the Components.
I'm good as long as long as it's properly documented, but this isn't a blocker for release, so all good.
I would recommend exploring or noting the possible approaches I mentioned, since it might be difficult for downstream users to see how to implement stateful parameter syncs.
All right, since you've got no more comments and examples of stateful adaptive Components are in the README, I'm closing this up.
I think there's definitely room for improving documentation and examples but I'm hoping that'll come along the road as this gets used.
If I understood correctly, this adaptive component is a way to expose an interface to swap between node components dynamically.
What is the intended interaction with the parameter system for the implemented nodes for the different architectures? Are parameters meant to be bound to the adaptive component (essentially the node manager), or is the intention to have the parameters bound to the architecture specific nodes instead?