Source fail to build on architectures without compiler support for atomic operations

[petterreinholdtsen]

When built on all the architectures in Debian, as can be seen on <URL: https://buildd.debian.org/status/package.php?p=restinio&suite=sid >, there are several build failures.

The issue at hand seem to be missing atomic support in the compiler. I found a similar problem and a proposed CMake fix in https://github.com/kimwalisch/primesieve/pull/46 .

[petterreinholdtsen]

https://bugs.debian.org/960824

[eao197]

@ngrodzitski could you take a look at this? I'm afraid I won't have time in the next two week :(

[petterreinholdtsen]

@ngrodzitski Hi. Any news on this? The problem block a new version of opendht and in turn jami in Debian from entering testing, and I hoped it could be solvable without Debian specific hacks. :)

[ngrodzitski]

Hi @petterreinholdtsen Proposed workaround looks good to me (it can be integrated in the same fashion like we do with link_threads_if_necessary). The only issue is how to test the fix, I have no any close environment. @eao197 if you ok with that approach, then I can be fix CMake build.

[eao197]

@ngrodzitski I don't have any objections.

[eao197]

Maybe a separate branch should be created for applying those changes and then this branch will be merged into the main 0.6-dev branch after testing?

[ngrodzitski]

Separate branch, sure.

[ngrodzitski]

@petterreinholdtsen Can you test https://github.com/Stiffstream/restinio/tree/0.6-fix-libatomic-issue-98, please?

[petterreinholdtsen]

[Nicolai Grodzitski]

@petterreinholdtsen Can you test https://github.com/Stiffstream/restinio/tree/0.6-fix-libatomic-issue-98, please?

I tried on mipsel, but the build failed.

    cd obj-mipsel-linux-gnu && cmake -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_BUILD_TYP

E=None -DCMAKE_INSTALL_SYSCONFDIR=/etc -DCMAKE_INSTALL_LOCALSTATEDIR=/var -DCMAKE_EXPO RT_NO_PACKAGE_REGISTRY=ON -DCMAKE_FIND_PACKAGE_NO_PACKAGE_REGISTRY=ON -DCMAKEINSTALL RUNSTATEDIR=/run "-GUnix Makefiles" -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_INSTALL_LIBDIR =lib/mipsel-linux-gnu -DRESTINIO_FIND_DEPS=1 -DRESTINIO_USE_EXTERNAL_HTTP_PARSER=1 -DR ESTINIO_FMT_HEADER_ONLY=0 -DRESTINIO_ALLOW_SOBJECTIZER=1 -DRESTINIOTEST=1 -DRESTINIO SAMPLE=1 -DRESTINIO_BENCH=1 -DCMAKE_LIBRARY_ARCHITECTURE=mipsel-linux-gnu ../dev CMake Warning (dev) in CMakeLists.txt: No project() command is present. The top-level CMakeLists.txt file must contain a literal, direct call to the project() command. Add a line of code such as

project(ProjectName)

near the top of the file, but after cmake_minimum_required().

CMake is pretending there is a "project(Project)" command on the first line. This warning is for project developers. Use -Wno-dev to suppress it.

-- The C compiler identification is GNU 9.3.0 -- The CXX compiler identification is GNU 9.3.0 -- Check for working C compiler: /usr/bin/cc -- Check for working C compiler: /usr/bin/cc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: /usr/bin/c++ -- Check for working CXX compiler: /usr/bin/c++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- Found Boost: /usr/include (found version "1.67.0") found components: regex -- Found Boost for regex. -- Found HTTP_Parser: /usr/include
-- http_parser libraries: /usr/lib/mipsel-linux-gnu/libhttp_parser.so -- http_parser include dir: /usr/include -- Found OpenSSL: /usr/lib/mipsel-linux-gnu/libcrypto.so (found version "1.1.1g")
OpenSSL include dir: /usr/include OpenSSL libraries: /usr/lib/mipsel-linux-gnu/libssl.so;/usr/lib/mipsel-linux-gnu/libcrypto.so -- Found PCRE: /usr/lib/mipsel-linux-gnu/libpcre.so
-- PCRE_LIBRARIES='/usr/lib/mipsel-linux-gnu/libpcre.so' -- PCRE_INCLUDE_DIRS='/usr/include' -- Found PCRE2: /usr/lib/mipsel-linux-gnu/libpcre2-8.so
-- PCRE2_LIBRARIES='/usr/lib/mipsel-linux-gnu/libpcre2-8.so' -- PCRE2_INCLUDE_DIRS='/usr/include' -- PCRE2_LIBRARIES='/usr/lib/mipsel-linux-gnu/libpcre2-8.so' -- PCRE2_INCLUDE_DIRS='/usr/include' CMake Error at CMakeLists.txt:167 (add_subdirectory): add_subdirectory given source "so_5" which is not an existing directory.

-- Found ZLIB: /usr/lib/mipsel-linux-gnu/libz.so (found version "1.2.11") -- ZLIB_LIBRARIES='/usr/lib/mipsel-linux-gnu/libz.so' -- ZLIB_INCLUDE_DIRS='/usr/include' -- RESTinio dependencies are included with find_package() -- Looking for pthread.h -- Looking for pthread.h - found -- Performing Test CMAKE_HAVE_LIBC_PTHREAD -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed -- Check if compiler accepts -pthread -- Check if compiler accepts -pthread - yes -- Found Threads: TRUE
-- Configuring incomplete, errors occurred! See also "/home/pere/restinio/obj-mipsel-linux-gnu/CMakeFiles/CMakeOutput.log". See also "/home/pere/restinio/obj-mipsel-linux-gnu/CMakeFiles/CMakeError.log". cd obj-mipsel-linux-gnu && tail -v -n +0 CMakeCache.txt ==> CMakeCache.txt <==

This is the CMakeCache file.

Not sure if you need the rest, which seem to be mostly recitations of various generated files. I am not skilled enough with cmake to have a guess on what is wrong.

This is using cmake 3.16.3-3, if that matters.

-- Happy hacking Petter Reinholdtsen

[eao197]

Looks like you did:

git clone https://.../restinio restinio
cd restinio
git checkout 0.6-fix-libatomic-issue-98

and then run CMake without acquiring RESTinio dependencies like SObjectizer. That is why CMake can't find so_5 subfolder.

The right way is to install Ruby and Mxx_ru RubyGem and then:

git clone https://.../restinio restinio
cd restinio
git checkout 0.6-fix-libatomic-issue-98
mxxruexternals

But I'm afraid the compilation of SObjectizer will fail because of the same error with std::atomic.

So try to disable usage of SObjectizer during the invocation of CMake: -DRESTINIO_ALLOW_SOBJECTIZER=OFF.

[ngrodzitski]

It expects dev/so_5 (https://github.com/Stiffstream/sobjectizer) which is obtained by mxxruexternals (a part of Mxx_ru - ruby-based build tool, it is a very simple package manager, and most likely it is absent on your test env). Sobjectizer is included in tarball for releases you can steal it from one of them to test the build.

[petterreinholdtsen]

I tested by fetching the build dependencies listed in the Debian package, and fetching the source provided by the restinio to do a test build. If other external sources are needed, perhaps someone can provide a tarball for me to test instead?

[eao197]

@petterreinholdtsen A tarball is in the attachment. restinio-libatomic-issue-98.tar.gz

[petterreinholdtsen]

I tested the tarball on a mipsel installation, and the build failed like this:

../../../so_5/libso_s.5.5.24.3.a(mbox_core.cpp.o): in function `std::__atomic_base<unsigned long long>::operator++()':
/usr/include/c++/9/bits/atomic_base.h:319: undefined reference to `__atomic_fetch_add_8'
/usr/bin/ld: /usr/include/c++/9/bits/atomic_base.h:319: undefined reference to `__atomic_fetch_add_8'
/usr/bin/ld: /usr/include/c++/9/bits/atomic_base.h:319: undefined reference to `__atomic_fetch_add_8'
/usr/bin/ld: /usr/include/c++/9/bits/atomic_base.h:319: undefined reference to `__atomic_fetch_add_8'
/usr/bin/ld: /usr/include/c++/9/bits/atomic_base.h:319: undefined reference to `__atomic_fetch_add_8'
/usr/bin/ld: ../../../so_5/libso_s.5.5.24.3.a(mbox_core.cpp.o):/usr/include/c++/9/bits/atomic_base.h:319: more undefined references to `__atomic_fetch_add_8' follow
collect2: error: ld returned 1 exit status
make[3]: *** [test/websocket/ws_connection/CMakeFiles/_unit.test.ws_connection.dir/build.make:90: test/websocket/ws_connection/_unit.test.ws_connection] Error 1

[eao197]

This problem doesn't relate to RESTinio itself, it's a problem with SObjectizer that is used in several tests and examples. Can you disable it by using argument -DRESTINIO_ALLOW_SOBJECTIZER=OFF with CMake?

[petterreinholdtsen]

Running cmake with that option provided the following feedback: RESTINIO_ALLOW_SOBJECTIZER can't be OFF if RESTINIO_TEST or RESTINIO_SAMPLE or RESTINIO_BENCH is used.

Changing the build to set the three variables to 0, made the build 'succeed', by simply doing nothing. Was this intentional?

[eao197]

Hm... It seems that we should update our sample/CMakeLists.txt and test/CMakeLists.txt to exclude some project files if RESTINIO_ALLOW_SOBJECTIZER is turned off.

I'll upload a new tarball when it will be done.

[eao197]

Here is updated tarball. restinio-libatomic-issue-98-20200720.tar.gz

If RESTINIO_ALLOW_SOBJECTIZER is set to OFF then tests, examples, and benchmarks that require SObjectizer won't be built.

[petterreinholdtsen]

[eao197]

Here is updated tarball. restinio-libatomic-issue-98-20200720.tar.gz

If RESTINIO_ALLOW_SOBJECTIZER is set to OFF then tests, examples, and benchmarks that require SObjectizer won't be built.

Using these flags

CMAKE_OPTIONS := -DRESTINIO_FIND_DEPS=1 \ -DRESTINIO_USE_EXTERNAL_HTTP_PARSER=1 \ -DRESTINIO_FMT_HEADER_ONLY=0 \ -DRESTINIO_ALLOW_SOBJECTIZER=1 \ -DRESTINIO_TEST=1 \ -DRESTINIO_SAMPLE=1 \ -DRESTINIO_BENCH=1 \ -DRESTINIO_ALLOW_SOBJECTIZER=OFF \ -DCMAKE_LIBRARY_ARCHITECTURE="$(DEB_HOST_MULTIARCH)"

I get this error when building using the Debian build rules:

/usr/bin/cmake -E cmake_link_script CMakeFiles/cmTC_2f499.dir/link.txt --verbose=1 /usr/bin/cc -g -O2 -fdebug-prefix-map=/home/pere/restinio-libatomic-issue-98=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -DCMAKE_HAVE_LIBC_PTHREAD -Wl,-z,relro -Wl,-z,now CMakeFiles/cmTC_2f499.dir/src.c.o -o cmTC_2f499 /usr/bin/ld: CMakeFiles/cmTC_2f499.dir/src.c.o: in function main': ./obj-mipsel-linux-gnu/CMakeFiles/CMakeTmp/./obj-mipsel-linux-gnu/CMakeFiles/CMakeTmp/src.c:11: undefined reference topthread_create' /usr/bin/ld: ./obj-mipsel-linux-gnu/CMakeFiles/CMakeTmp/./obj-mipsel-linux-gnu/CMakeFiles/CMakeTmp/src.c:11: undefined reference to pthread_create' /usr/bin/ld: ./obj-mipsel-linux-gnu/CMakeFiles/CMakeTmp/./obj-mipsel-linux-gnu/CMakeFiles/CMakeTmp/src.c:12: undefined reference topthread_detach' /usr/bin/ld: ./obj-mipsel-linux-gnu/CMakeFiles/CMakeTmp/./obj-mipsel-linux-gnu/CMakeFiles/CMakeTmp/src.c:12: undefined reference to pthread_detach' /usr/bin/ld: ./obj-mipsel-linux-gnu/CMakeFiles/CMakeTmp/./obj-mipsel-linux-gnu/CMakeFiles/CMakeTmp/src.c:13: undefined reference topthread_join' /usr/bin/ld: ./obj-mipsel-linux-gnu/CMakeFiles/CMakeTmp/./obj-mipsel-linux-gnu/CMakeFiles/CMakeTmp/src.c:13: undefined reference to `pthread_join' collect2: error: ld returned 1 exit status gmake[3]: *** [CMakeFiles/cmTC_2f499.dir/build.make:87: cmTC_2f499] Error 1 gmake[3]: Leaving directory '/home/pere/restinio-libatomic-issue-98/obj-mipsel-linux-gnu/CMakeFiles/CMakeTmp'

-- Happy hacking Petter Reinholdtsen

[eao197]

It looks like Debian on MIPS lacks not only std::atomic but also std::thread. Or there are some specific compiler/linker options for proper linking with the implementation of POSIX threads.

Anyway, without an access to that platform, we can't play with compiler/linker options. So I have no idea now how this issue can be solved.

[petterreinholdtsen]

[eao197]

It looks like Debian on MIPS lacks not only std::atomic but also std::thread. Or there are some specific compiler/linker options for proper linking with the implementation of POSIX threads.

Using threads require -lpthread to find the POSIX thread methods.

My guess is that this library normally would be pulled in by some other library dependency, which is no longer included now. Perhaps sobjectizer would normally pull in libpthread?

-- Happy hacking Petter Reinholdtsen

[eao197]

Linking to pthreads is automatically performed for samples and unit-tests because of call of link_threads_if_necessary in the corresponding CMake files:

https://github.com/Stiffstream/restinio/blob/3d4a2dc82a66bc324abe317d4798824e83f64fba/dev/cmake/unittest.cmake#L15

https://github.com/Stiffstream/restinio/blob/accf6268cdc21e2063eccea8c45746da65aab4bc/dev/cmake/sample.cmake#L14

So I suppose there is some issue with detecting of pthreads by CMake.

The link_threads_if_necessary is implemented here: https://github.com/Stiffstream/restinio/blob/accf6268cdc21e2063eccea8c45746da65aab4bc/dev/cmake/link_threads_if_necessary.cmake#L1-L8

[petterreinholdtsen]

So, how can the issue be solved then?

[eao197]

As I said earlier:

Anyway, without an access to that platform, we can't play with compiler/linker options. So I have no idea now how this issue can be solved.

[petterreinholdtsen]

[eao197]

As I said earlier:

Anyway, without an access to that platform, we can't play with compiler/linker options. So I have no idea now how this issue can be solved.

Right. My best idea there is to set up a mipsel instance in qemu. I only have access to the Debian porter boxes myself, and can not grant access to these.

-- Happy hacking Petter Reinholdtsen

[eao197]

My best idea there is to set up a mipsel instance in qemu.

Is there any description of how it can be done?

[petterreinholdtsen]

[eao197]

My best idea there is to set up a mipsel instance in qemu.

Is there any description of how it can be done?

A quick search for 'install debian mipsel qemu' sent me to <URL: https://gist.github.com/mzpqnxow/862b38e920a9c47f21197b3fc2008e52 > which seem like a fair recipe. You should switch the image to Debian 10 or 11 instead of 9, but the rest look like it would work.

-- Happy hacking Petter Reinholdtsen

[eao197]

Thanks. I'll try to take a look at it.

[eao197]

I've tried to compile an example from RESTinio in Debian 10 inside QEMU (installed as described above) and didn't get any linker errors:

eao197@debian:~/sandboxes/restinio/dev/cmake_build$ cmake --build . --target sample.hello_world_basic
[ 50%] Built target http_parser
Scanning dependencies of target sample.hello_world_basic
[ 50%] Building CXX object sample/hello_world_basic/CMakeFiles/sample.hello_world_basic.dir/main.cpp.o
In file included from /home/eao197/sandboxes/restinio/dev/restinio/http_headers.hpp:11:0,
                 from /home/eao197/sandboxes/restinio/dev/restinio/request_handler.hpp:15,
                 from /home/eao197/sandboxes/restinio/dev/restinio/settings.hpp:18,
                 from /home/eao197/sandboxes/restinio/dev/restinio/all.hpp:14,
                 from /home/eao197/sandboxes/restinio/dev/sample/hello_world_basic/main.cpp:3:
/home/eao197/sandboxes/restinio/dev/restinio/impl/include_fmtlib.hpp:24:32: warning: unknown option after ‘#pragma GCC diagnostic’ kind [-Wpragmas]
 #pragma GCC diagnostic ignored "-Wimplicit-fallthrough"
                                ^~~~~~~~~~~~~~~~~~~~~~~~
In file included from /home/eao197/sandboxes/restinio/dev/restinio/impl/connection.hpp:109:0,
                 from /home/eao197/sandboxes/restinio/dev/restinio/impl/acceptor.hpp:15,
                 from /home/eao197/sandboxes/restinio/dev/restinio/http_server.hpp:14,
                 from /home/eao197/sandboxes/restinio/dev/restinio/all.hpp:17,
                 from /home/eao197/sandboxes/restinio/dev/sample/hello_world_basic/main.cpp:3:
/home/eao197/sandboxes/restinio/dev/restinio/impl/parser_callbacks.ipp: In function ‘int restinio::impl::restinio_chunk_header_cb(http_parser*)’:
/home/eao197/sandboxes/restinio/dev/restinio/impl/parser_callbacks.ipp:163:26: warning: narrowing conversion of ‘parser->http_parser::content_length’ from ‘uint64_t {aka long long unsigned int}’ to ‘std::size_t {aka unsigned int}’ inside { } [-Wnarrowing]
     std::size_t{ parser->content_length } );
                  ~~~~~~~~^~~~~~~~~~~~~~
[100%] Linking CXX executable sample.hello_world_basic
[100%] Built target sample.hello_world_basic
eao197@debian:~/sandboxes/restinio/dev/cmake_build$ 

The latest stable version (0.6.9) of RESTinio was used. This example (hello_world_basic) uses std::thread inside on_thread_pool https://github.com/Stiffstream/restinio/blob/0ef04a26155d8aa4a56a36cc013c7e722675da21/dev/sample/hello_world_basic/main.cpp#L29