Migration to WebRTC M86/Ubuntu 20.04/ROS Noetic

[kubja]

First of all, thanks to the author @mitchellwills and maintainers @rctoris and @roehling of this package to make WebRTC available in the ROS ecosystem. I know how frustrating it can be to work with Chromium projects.

This pull request migrates the webrtc package from Chromium 59 released in 2017 to M86, the latest stable version at this time. Since 2017, the project went through active development, security patches, and the first stable release.

Highlights:

• The WebRTC library APIs has changed significantly since then, so this pull request does cover changes in most of the webrtc_ros package. The major change was that cricket::VideoCapturer has been out phased and for this, the RosVideoCapturer had to be rewritten. Another significant change was in threading and how rtc::Thread is created.
• The file structure of Chromium was changed, so there are some changes in the webrtc package CMake project mostly related to installing the header files and libraries.
• The webrtc_ros had to be migrated to C++ 14 standard as the WebRTC library does not anymore support C++ 11.
• I removed RosLogContext as it seems to be unused at the latest implementation and webrtc::TraceCallback has been changed.
• I also migrated to Ubuntu 20.04 using suggested changes from @GMahmoud in issue #51

I am including a Dockefile for testing in Ubuntu 20.04/ ROS Noetic environment. You can use it as a reference for dependencies and possibly use it to build the packages by running :

docker build -t ros:webrtc .

I welcome any feedback. I would like to work with maintainers to have this merged and released for Noetic. Current blockers:

• I could not get libjsoncpp compiled with the webrtc project. The official Debian package is compiled with the wrong ABI so currently I compile the libjsoncpp from the source and link it from /usr/lib/ instead of how it was done previously.
• Currently, async_web_server_cpp has not been released for ROS noetic so it is compiled from source.

[flynneva]

trying to help out and test your branch on Ubuntu 20.04/Noetic.

running into an error after running catkin_make_isolated:

ERROR at //build/config/BUILDCONFIG.gn:402:1: set_sources_assignment_filter argument must be an empty list.
set_sources_assignment_filter(sources_assignment_filter)
^------------------------------------------------------
CMake Error at CMakeLists.txt:66 (message):
  cannot prepare WebRTC build system

any tips?

UPDATE: found this documentation stating that this function is no-longer used...? not quite sure how to remove it from being used though...

UPDATE2: definitely not what you are supposed to do but I commented out lines 316-385 in the BUILDCONFIG.gn file mentioned in the error and the build continued on.

[flynneva]

@kubja just made a PR to your branch here where I think i fixed the jsoncpp linking issue.....go ahead and test it out to make sure though.

[kubja]

Awesome, thanks for testing this! I'll checkout your pull request later today.

Regarding set_sources_assignment_filter(sources_assignment_filter). Could that come from the prebuilt old version of gn coming with the package ? Strange this doesn't happen with the docker environment. I just realized I'm copying out depot tools from my environment. I will try with dockerignore of that.

[flynneva]

I was thinking the same thing but unfortunately removing the prebuilts, forcing it to clone GN and build from source but I still saw the issue.

[kubja]

I was thinking the same thing but unfortunately removing the prebuilts, forcing it to clone GN and build from source but I still saw the issue.

I rebuilt in the docker environment with dockerignore on depot_tools and it still builds correctly. I investigated a bit and my version actually doesn't use the prebuilt gn from webrtc/prebuilt/ but the one fetched from the depot_tools repository fetched with get_depot_tools script. @flynneva, could you please check that the latest depot_tools were pulled on your side? When I run in webrtc/build/webrtc/src:

../../depot_tools/gn --version

I get 1819 (e327ffdc).

[roehling]

Thank you for this awesome pull request! At first glance, your changes look good to me. Since the current build in develop is broken anyway due to the changes in depot_tools, I suggest I'll merge this PR as-is, and then we iron out the remaining problems in follow-up PRs, particularly the libjsoncpp issue. For the async_web_server_cpp, we'll have to revive GT-RAIL/async_web_server_cpp; I don't have write access to that repository, so we'll have to find someone who has. Failing that, we can try to fork the repository and hijack the package, but I'd like to avoid that if possible.

[flynneva]

@kubja when I run the command you said i get 1868 (646a62e0)

[GMahmoud]

Hello guys :) @kubja @flynneva I am not sure if my following script can help you with the gn issue but few months ago I got some problems as well with gn on my own webrtc project but unfortunately I do not remember the exact message. So what I did I deleted the the gn coming from the depot_tools and rebuild it from source. It's working for ubuntu 20.04.

I applied some modifications on this file https://github.com/RobotWebTools/webrtc_ros/blob/develop/webrtc/build/get_gn

#!/bin/bash
set -e
rootdir="$(cd "$(dirname "$0")"; pwd)"
export PATH="${rootdir}/depot_tools:$PATH"
cd "$rootdir/webrtc/src"
use_source=1
commit="92116c5af6d9845b6337d52b001a148c92f62418"

if [ "$use_source" -eq 1 ]
then
        if [ ! -d "$rootdir/webrtc/src/third_party/gn" ]
        then
            echo "Building GN from source"
            git clone https://gn.googlesource.com/gn third_party/gn
        fi

        cd third_party/gn
        current_commit=$(git rev-parse HEAD)
        if [[ ! "$current_commit" == "$commit" ]]
        then
            rm gn -f
            git checkout ${commit}
            CC=gcc CXX=g++ LDFLAGS=-fuse-ld=gold python build/gen.py
            "$rootdir/depot_tools/ninja" -C out gn
            cp out/gn .
        fi

        rm "$rootdir/depot_tools/gn.py" -f
        rm "$rootdir/depot_tools/gn" -f 

        cp out/gn "$rootdir/depot_tools/"

elif ! gn --help &>/dev/null
then
    mkdir -p third_party/gn
    a="$(dpkg-architecture -qDEB_HOST_ARCH || uname -m)"
    case "$a" in
        i?86)
            cp "$rootdir/prebuilt/gn-linux32" third_party/gn/gn
        ;;
        x86_64|amd64)
            cp "$rootdir/prebuilt/gn-linux64" third_party/gn/gn
        ;;
        armhf|armv*)
            cp "$rootdir/prebuilt/gn-linux_armhf" third_party/gn/gn
        ;;
        arm64|aarch64)
            cp "$rootdir/prebuilt/gn-linux_arm64" third_party/gn/gn
        ;;
        *)
            echo "WARNING: unknown target platform: $a"
        ;;
    esac
    if ! gn --help &>/dev/null
    then
        rm -rf third_party/gn
        echo "Prebuilt GN is not working, building from source"
        git clone https://gn.googlesource.com/gn third_party/gn
        cd third_party/gn
        CC=gcc CXX=g++ LDFLAGS=-fuse-ld=gold python build/gen.py
        "$rootdir/depot_tools/ninja" -C out gn
        cp out/gn .
    fi
fi

@flynneva could you test this please on your machine ? I will fetch @kubja modifications soon and give my opinion

Cheers :smile:

[roehling]

After waiting two months and trying to contact a bunch of people via email, I decided to fork async_web_server_cpp. The Noetic release is pending.