search

lf-lang / lingua-franca

Intuitive concurrent programming in any language
https://www.lf-lang.org
Other
235 stars 63 forks source link

Enhance custom build process for federated execution #309

Open lsk567 opened 3 years ago

lsk567 commented 3 years ago

Learn from ROS colcon and design a build strategy for distributed systems. Automate the workflow @rcakella is using.

One idea: We need to extend the build target property to help execute different commands for different federates. We can make build take a dictionary that takes a federate name to a list of strings. We need to ensure that the federate names match. There seems to be something subtle with the xtext infrastructure.

Another idea: Rely on some external build system.

Soroosh129 commented 3 years ago

This is a very interesting proposal.

I think with ROS and colcon, the main target use-case is to build a ROS package with multiple nodes on the same machine. The actual build itself is done using ament-cmake (by default, this is configurable), which closely resembles the way the C++ target builds LF programs. Does this suggest that we might be better off using cmake?

One thing that I like about our current implementation is that if a federate is remote, the code generator currently will not compile the code for that federate. Instead, it will put the required compile command in a bash script with a *_distribute.sh name. The bash script will SSH to the remote machine and compile the code there. This feature enabled me to create a distributed application where one of the nodes was running on an ARM-based processor. I don't know if that is possible with cmake. We might need to create a CMakeLists.txt for each federate perhaps?

Also, can you please provide a code sketch of the solution you are proposing?

lsk567 commented 3 years ago

Thanks Soroush for the insight and sorry for missing this message earlier. This feature was inspired by @rcakella 's effort in building a distributed LF application. In his case, each machine needs a different way to build the federate and he created a workflow to automate this process. (I am currently not too familiar with the exact approach but it seems like some XML configuration files were involved. @rcakella Can you provide more details on this and please correct me if I am wrong?) In general, I think a solution that includes distributing federate code and executing custom build commands for each federate is needed to easily create a distributed application that involves different environments.

How much of this workflow can colcon do? Can colcon build for multiple nodes in a distributed setting?

rcakella commented 3 years ago

Thanks @lsk567 for raising this issue and sorry for the late comment. Here is the current flow I adopted. I utilized LF C target to create a ROS2 package (similar to steps in https://github.com/icyphy/lingua-franca/blob/master/example/C/ROS/build-ROS-node.sh) so as to interact with other (non-LF) ROS2 nodes on a remote server. This LF generated package gets compiled using Colcon along with other ROS2 packages by running a shell script manually on this remote server. The XML files Shaokai mentioned are for package description following the usual Colcon build process. For my specific distributed application, a docker compose file was used to launch specific ROS2 nodes as containers that interact over DDS.

I think if we were to automate this process for a build property ("ros"), simple first step may be to autogenerate the Colcon package structure (along with required config files) that includes the generated code. Second step may be to copy over the generated package to the specified remote host at the specified path (similar to how federated code is currently deployed). Third, invoke a shell script (with a unique name) that jointly builds all ROS2 packages. Finally, run another shell script or rely on roslaunch for execution.