search

ros-tooling / cross_compile

A tool to build ROS and ROS2 workspaces for various targets
Apache License 2.0
188 stars 60 forks source link

Cross Compile: Sysroot creator script update #20

Closed piraka9011 closed 5 years ago

piraka9011 commented 5 years ago

This commit updates the cross compilation (CC) sysroot creator script based on the design requirements defined in this ROS2 PR. Colcon mixins and unit tests will be merged in a later PR which can be found in the colcon-mixins branch and tests branch respectively.

The script uses a base Docker image to cross compile ROS2 packages across different architectures based on the arguments provided to it. Documentation on how to use is provided.

The script was tested on MacOS Mojave (10.14), Ubuntu Bionic (18.04) for x86 and aarch64, as well as Ubuntu Xenial (16.04).

Edit: Requesting review from @filiperinaldi @christianrauch

alsora commented 5 years ago

Hi @piraka9011

Could you clarify me some doubts about this cross-compilation framework?

From what I'm seeing, the cross-compilation happens inside a Docker image built with qemu. We have a base image of the target architecture, referred to as ROS2_BASE_IMG, and an additional image defined by Dockerfile_workspace script. This has the following issues:

Moreover, it would be nice to provide instructions for doing the following:

Besides this, I think that having cross-compilation embedded in colcon will be much nicer than the scripts that I wrote for the other PR https://github.com/ros2/cross_compile/pull/18, but with the condition that it allows easy customization for different tools (i.e. with or without Docker) and architectures.

piraka9011 commented 5 years ago

Hey @alsora!

I'll do my best to address your concerns but most of them go back to the design discussion in the referenced PR.

The cross-compilation will be much slower than doing it in a X86 environment.

and

I think that having cross-compilation embedded in colcon will be much nicer than the scripts that I wrote for the other PR

Dirk and Tully have stated in the design PR that they does not want this to be part of Colcon:

Dirk:

This goes against one of the stated software design goals of colcon:

Each functionality added should follow the principle “you don’t pay for what you don’t use”.

So if a plugin proving support for X doesn't need sysroot support I shouldn't need to implement it.

Tully:

I suggest having the platform support builds embedded into docker images instead of on the host. So that you don't need to have the cross compilation resources on your local machine, just a docker dependency.

What if, for a specific architecture, it is not possible to run all the instructions in Dockerfile_workspace?

We are assuming that:

  1. All commands are specific to building ROS2 packages only.
  2. All the packages can be built for the specified architecture.
  3. We aren't using any arcane/complex tools.

The Dockerfile_workspace has been tested for various architectures and our unit tests (including them after this PR) should indicate whenever any regressions occur.

Build without having a ROS2_BASE_IMG, but rather a sysroot directory

Again, back to Tully's comment :)

I suggest having the platform support builds embedded into docker images instead of on the host. So that you don't need to have the cross compilation resources on your local machine, just a docker dependency.

Build only a specific package

This could be a simple addition actually, passing the packages as a list to the Docker image generating the sysroot. But then again, one could just COLCON_IGNORE packages or only include the packages they want to compile in their ros2_ws/src under the generated sysroot.

I'd recommend suggesting this in the design PR and we'll update accordingly.

alsora commented 5 years ago

Just some quick replies, then tomorrow I will read the design PR more in details.

I suggest having the platform support builds embedded into docker images instead of on the host. So that you don't need to have the cross compilation resources on your local machine, just a docker dependency.

I agree with this statement, but there are different ways for satisfying it:

Correct me if I'm wrong, but you did the first. I think that the second implementation would be much better for the reasons mentioned above.

Build without having a ROS2_BASE_IMG, but rather a sysroot directory

This does not require to install any dependency on the host system in order to be satisfied. It may be the case that I do not have a Docker image at hand for my target architecture, but rather I only have the lib, include, etc directories for my target system, downloaded from somewhere.

IMO, the cross-compilation command/script should accept the following arguments:

Build only a specific package

only include the packages they want to compile in their ros2_ws/src under the generated sysroot.

This is exactly what I meant, however this will require that somewhere the whole ROS2 SDK is present. It is not clear to me how it is possible to do that

piraka9011 commented 5 years ago

Thanks for the clarification.

Regarding:

Use a Docker image for Ubuntu 18.04 with all the cross-compilation dependencies and cross-compile ROS2 there.

Yes, adding the cross compilation dependencies inside the Docker image could also be a viable alternative. We would have to provide our own images for this though which is something our team can't do at the moment and is why we went with the first. (We are working with our legal team on this).

Regarding adding the option to select your sysroot, that is also something we should add to the design docs and then work on implementing from there :)

ruffsl commented 5 years ago

We would have to provide our own images for this

Why? Are we relying on cross compilers that can not be installed via apt in a Dockerfile?

piraka9011 commented 5 years ago

@ruffsl, just re-read and realized we aren't distributing/hosting any images so scratch my last comment.

Yep, we can just apt install the packages as well which as valid option. I believe this was addressed in the design doc as a feature to iterate on. Line 89 under 'Proposed Approach' where we have a docker image with all the compilation requirements.

alsora commented 5 years ago

@piraka9011 have you had a look at the Dockefiles that I'm using in my PR? Specifically, the ones contained in the docker_environments directory. They are used exactly for this purpose.

There is a dockerfile_base that installs general ROS2 dependencies in a x86 Ubuntu 18.04 environment. Then you have a different file for each architecture that installs the compiler.

Do you. Think that we could integrate them into this PR?

ruffsl commented 5 years ago

There is a dockerfile_base that installs general ROS2 dependencies in a x86 Ubuntu 18.04 environment.

FYI, over on osrf/docker_images , we have a osrf/ros2:devel tag that is hosted for this purpose to build the latest branches of ros2 from source. In fact, its used as a base image for the source Dockerfile as well. A rebuild on Docker Hub for the devel tag is also triggered daily to keep it just as fresh as our nightly images as well.

emersonknapp commented 5 years ago

@piraka9011 @thomas-moulard how are we doing on this tool, are we ready to move forward with this first version or does it need reworking?

thomas-moulard commented 5 years ago

We agreed to basically merge our fork in this repo, as long as 1/ the current state of master is tagged so that it remains accessible, and 2/ we update the doc as needed. My suggestion is to close this PR, prepare a doc PR update, and just merged directly back our fork into master after tagging the current HEAD.

piraka9011 commented 5 years ago

ditto Thomas + I believe we just need to x2 check the requirements in the setup.py to make sure it can be bloom released/installed as a standalone package.

prajakta-gokhale commented 5 years ago

Closing in favor of https://github.com/ros-tooling/cross_compile/pull/22