Build on Ubuntu 20.04

[srmainwaring]

@gnanukoth I've posted your request here in case others are interested. It turns out to be a bit more complicated on 20.04, mainly because the default versions of some of the dependencies need to be upgraded. I've run the steps below in an Ubuntu 20.04 VM and they seem to work.

I saw that you had raised an issue with the gz-transport repo, following my request for python bindings under the gz-sim repo. I saw your gz-python implementation and tried following your instructions. My system is Ubuntu 20.04 and I had default python 3.8 installed at the system level. But the cmake threw an error as it expected python3.10. So, I installed python 3.10 and configured it to be the default python version. But when I run cmake again for gz-python, it says there is some faulty installation for python 3.10. I am not sure how to handle this or what the problem is. Can you provide any insights for the same? I have attached the error that I got when running the command, for your reference.

Expand details to see build log.

```bash -- Build protobuf: ON -- Build pybind11_protobuf: ON -- Could NOT find Python (missing: Python_INCLUDE_DIRS Python_LIBRARIES Development Development.Module Development.Embed) (found suitable exact version "3.10.10") -- Compiling against Gazebo Garden -- Looking for gz-msgs9 -- found version 9.3.0 -- Searching for dependencies of gz-msgs9 -- Found Protobuf: /usr/lib/x86_64-linux-gnu/libprotobuf.so;-lpthread (found suitable version "3.6.1", minimum required is "3") -- Looking for gz-math7 -- found version 7.1.0 -- Searching for dependencies of gz-math7 -- Looking for gz-utils2 -- found version 2.0.0 -- Searching for dependencies of gz-utils2 -- Checking for module 'tinyxml2' -- Found tinyxml2, version 6.2.0 -- Looking for gz-transport12 -- found version 12.1.0 -- Searching for dependencies of gz-transport12 -- Config-file not installed for ZeroMQ -- checking for pkg-config -- Checking for module 'libzmq >= 4' -- Found libzmq , version 4.3.2 -- Checking for module 'uuid' -- Found uuid, version 2.34.0 -- Looking for gz-utils2 -- found version 2.0.0 -- Searching for dependencies of gz-utils2 -- Searching for component [cli] -- Looking for gz-utils2-cli -- found version 2.0.0 -- Searching for dependencies of gz-utils2-cli -- Looking for gz-msgs9 -- found version 9.3.0 -- Fetching protobuf -- -- 3.21.5.0 -- Fetching protobuf - fetched -- Fetching pybind11_protobuf -- Build abseil-cpp: ON -- Build protobuf: ON -- Python: Build pybind11: ON -- Could NOT find Python (missing: Python_INCLUDE_DIRS Python_LIBRARIES Interpreter Development Development.Module Development.Embed) -- Fetching Abseil-cpp CMake Warning at build/_deps/absl-src/CMakeLists.txt:70 (message): A future Abseil release will default ABSL_PROPAGATE_CXX_STD to ON for CMake 3.8 and up. We recommend enabling this option to ensure your project still builds correctly. -- Fetching Abseil-cpp - fetched -- Fetching protobuf -- Fetching protobuf - fetched -- Fetching pybind11 -- pybind11 v2.10.1 -- Fetching pybind11 - fetched -- Found Protobuf: /usr/lib/x86_64-linux-gnu/libprotobuf.so;-lpthread (found version "3.6.1") -- Fetching pybind11_protobuf - fetched pybind11_protobuf_FOUND: pybind11_protobuf_SOURCE_DIR: /home/anna/gz_ws/src/gz-python/build/_deps/pybind11_protobuf-src pybind11_protobuf_INCLUDE_DIRS: -- Configuring done CMake Error: The following variables are used in this project, but they are set to NOTFOUND. Please set them or make sure they are set and tested correctly in the CMake files: Python_LIBRARY linked by target "pybind11_wrapped_proto_caster" in directory /home/anna/gz_ws/src/gz-python/build/_deps/pybind11_protobuf-src linked by target "pybind11_native_proto_caster" in directory /home/anna/gz_ws/src/gz-python/build/_deps/pybind11_protobuf-src linked by target "pybind11_proto_utils" in directory /home/anna/gz_ws/src/gz-python/build/_deps/pybind11_protobuf-src CMake Error in CMakeLists.txt: Imported target "pybind11::module" includes non-existent path "/usr/include/python3.10" in its INTERFACE_INCLUDE_DIRECTORIES. Possible reasons include: * The path was deleted, renamed, or moved to another location. * An install or uninstall procedure did not complete successfully. * The installation package was faulty and references files it does not provide. CMake Error in CMakeLists.txt: Imported target "pybind11::module" includes non-existent path "/usr/include/python3.10" in its INTERFACE_INCLUDE_DIRECTORIES. Possible reasons include: * The path was deleted, renamed, or moved to another location. * An install or uninstall procedure did not complete successfully. * The installation package was faulty and references files it does not provide. CMake Error in build/_deps/pybind11_protobuf-src/CMakeLists.txt: Imported target "pybind11::pybind11" includes non-existent path "/usr/include/python3.10" in its INTERFACE_INCLUDE_DIRECTORIES. Possible reasons include: * The path was deleted, renamed, or moved to another location. * An install or uninstall procedure did not complete successfully. * The installation package was faulty and references files it does not provide. CMake Error in build/_deps/pybind11_protobuf-src/CMakeLists.txt: Imported target "pybind11::module" includes non-existent path "/usr/include/python3.10" in its INTERFACE_INCLUDE_DIRECTORIES. Possible reasons include: * The path was deleted, renamed, or moved to another location. * An install or uninstall procedure did not complete successfully. * The installation package was faulty and references files it does not provide. CMake Error in build/_deps/pybind11_protobuf-src/CMakeLists.txt: Imported target "pybind11::pybind11" includes non-existent path "/usr/include/python3.10" in its INTERFACE_INCLUDE_DIRECTORIES. Possible reasons include: * The path was deleted, renamed, or moved to another location. * An install or uninstall procedure did not complete successfully. * The installation package was faulty and references files it does not provide. -- Generating done CMake Generate step failed. Build files cannot be regenerated correctly. ```

There are a few issues building on Ubuntu 20.04:

1. The default version of cmake is 3.16 not 3.18.
2. The default version of protoc is 3.6 not 3.21.5
3. gz-python expects Python version 3.10

update cmake

The cmake version is imposed by the dependency on one of the protobuf libraries. This SO overflow entry has a helpful summary of how to build cmake from source: https://askubuntu.com/questions/355565/how-do-i-install-the-latest-version-of-cmake-from-the-command-line.

$ mkdir cmake_ws && cd cmake_ws
$ version=3.18
$ build=1
$ wget https://cmake.org/files/v$version/cmake-$version.$build.tar.gz
$ tar -xzvf cmake-$version.$build.tar.gz
$ cd cmake-3.18.1/
$ ./bootstrap 
$ make -j4
$ sudo make install
$ cmake --version

update protoc

Here's a gist advising how to update the protoc version: https://gist.github.com/jambonn/1f5fffc23f97f8413372a438739c1bff. The tag to use is v21.5.

$ mkdir proto_ws && cd proto_ws
$ git clone https://github.com/google/protobuf.git
$ git checkout tags/v21.5
$ mkdir build && cd build
$ cmake .. -DCMAKE_CXX_FLAGS="-fPIC"
$ make -j4
$ sudo make install
$ protoc --version

build gz-sim7 from source

Needed because gz-msgs9 uses find_package() for the protobuf dependencies rather than fetch content (which is why we needed to upgrade protoc in the previous step). To check things are working you only need to build up to gz-transport12:

Remember to set:

$ export PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python

before starting the build to ensure the Python marshalling is used.

colcon build --symlink-install --merge-install --cmake-args -DCMAKE_BUILD_TYPE=RelWithDebInfo --packages-up-to gz-transport12

build gz-python

You can set the Python version to 3.8 in CMakeLists.txt. Version 3.10 is not required. It is set here because on macOS brew installs multiple Python versions side by side, and you can wind up with a mixture of versions during a colcon build if you are not careful.

[gnanukoth]

Noted, Thanks for support! Changing the python requirement to 3.8 helped complete the cmake build. Previously I had installed cmake version 3.25.2, since 3.16 was not compatible. Although the protoc version is 3.6 in my system, it did not throw any error. I will go ahead and update it to 3.21.5 as you have mentioned. But I would like to clarify if it is a necessity to downgrade cmake to 3.18; will the higher version cause any compatibility issues?

[gnanukoth]

colcon build --symlink-install --merge-install --cmake-args -DCMAKE_BUILD_TYPE=RelWithDebInfo --packages-up-to gz-transport12 May I know the directory in which the colcon build for the gz-sim7 has to be run? I am sorry, I am still not quite familiar with the source builds :( I currently have a binary install of gazebo garden in my system

[srmainwaring]

will the higher version cause any compatibility issues?

No, having a later version of cmake should not be an issue.

May I know the directory in which the colcon build for the gz-sim7 has to be run? I

Run colcon from the workspace directory (i.e. gz_garden_ws in the diagram below):

gz_garden_ws
├── build
├── install
└── src
    ├── gz-cmake
    ├── gz-common
    ├── gz-fuel-tools
    ├── gz-gui
    ├── gz-launch
    ├── gz-math
    ├── gz-msgs
    ├── gz-physics
    ├── gz-plugin
    ├── gz-python
    ├── gz-rendering
    ├── gz-sensors
    ├── gz-sim
    ├── gz-tools
    ├── gz-transport
    ├── gz-tutorials
    ├── gz-utils
    └── sdformat

I currently have a binary install of gazebo garden in my system

That may cause conflicts - check your search paths to make sure that headers and libraries are found in the source folder first rather than the system install.

The log you sent me indicates that the system headers are found first:

In file included from /usr/include/gz/msgs9/gz/msgs/clock.pb.h:17,
                 from /home/anna/workspace/src/gz-transport/src/Clock.cc:17:

This is a problem for gz-msgs as the C++ header files for the protobuf messages are generated using the older version of protoc.

[gnanukoth]

Noted, I will check that! Thank you so much for your support!

[gnanukoth]

I tried uninstalling the binary installation, but the gz folder in the usr/include still remains. How do I check or set correct search paths?