zeek / spicy

C++ parser generator for dissecting protocols & files.
https://docs.zeek.org/projects/spicy
Other
248 stars 37 forks source link

Failed to compile using GCC 11.2.1 #1891

Open AmazingPP opened 2 days ago

AmazingPP commented 2 days ago

The document states that only GCC >= 9 is required.

https://github.com/zeek/spicy/blob/ab69192894caef32e7bca316ec6fa5e65d4a40a4/doc/installation.rst#L210

Error log:

/spicy/spicy/runtime/src/base64.cc: In member function ‘hilti::rt::Bytes spicy::rt::base64::Stream::encode(const hilti::rt::Bytes&)’:
/spicy/spicy/runtime/src/base64.cc:38:40: error: call of overloaded ‘basic_string(SafeInt<long unsigned int, hilti::rt::integer::detail::SafeIntException>, <brace-enclosed initializer list>)’ is ambiguous
   38 |     std::string buf(data.size() * 2, {});
      |                                        ^
In file included from /opt/rh/devtoolset-11/root/usr/include/c++/11/string:55,
                 from /opt/rh/devtoolset-11/root/usr/include/c++/11/stdexcept:39,
                 from /spicy/hilti/runtime/include/hilti/rt/types/bytes.h:7,
                 from /spicy/spicy/runtime/src/base64.cc:3:
/opt/rh/devtoolset-11/root/usr/include/c++/11/bits/basic_string.h:3734:9: note: candidate: ‘std::basic_string<_CharT, _Traits, _Alloc>::basic_string(_InputIterator, _InputIterator, const _Alloc&) [with _InputIterator = SafeInt<long unsigned int, hilti::rt::integer::detail::SafeIntException>; _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>]’
 3734 |         basic_string(_InputIterator __beg, _InputIterator __end,
      |         ^~~~~~~~~~~~
/opt/rh/devtoolset-11/root/usr/include/c++/11/bits/basic_string.h:3673:7: note: candidate: ‘std::basic_string<_CharT, _Traits, _Alloc>::basic_string(std::basic_string<_CharT, _Traits, _Alloc>::size_type, _CharT, const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>; std::basic_string<_CharT, _Traits, _Alloc>::size_type = long unsigned int]’
 3673 |       basic_string(size_type __n, _CharT __c, const _Alloc& __a = _Alloc())
      |       ^~~~~~~~~~~~
/spicy/spicy/runtime/src/base64.cc: In member function ‘hilti::rt::Bytes spicy::rt::base64::Stream::decode(const hilti::rt::Bytes&)’:
/spicy/spicy/runtime/src/base64.cc:63:40: error: call of overloaded ‘basic_string(SafeInt<long unsigned int, hilti::rt::integer::detail::SafeIntException>, <brace-enclosed initializer list>)’ is ambiguous
   63 |     std::string buf(data.size() * 2, {});
      |                                        ^
In file included from /opt/rh/devtoolset-11/root/usr/include/c++/11/string:55,
                 from /opt/rh/devtoolset-11/root/usr/include/c++/11/stdexcept:39,
                 from /spicy/hilti/runtime/include/hilti/rt/types/bytes.h:7,
                 from /spicy/spicy/runtime/src/base64.cc:3:
/opt/rh/devtoolset-11/root/usr/include/c++/11/bits/basic_string.h:3734:9: note: candidate: ‘std::basic_string<_CharT, _Traits, _Alloc>::basic_string(_InputIterator, _InputIterator, const _Alloc&) [with _InputIterator = SafeInt<long unsigned int, hilti::rt::integer::detail::SafeIntException>; _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>]’
 3734 |         basic_string(_InputIterator __beg, _InputIterator __end,
      |         ^~~~~~~~~~~~
/opt/rh/devtoolset-11/root/usr/include/c++/11/bits/basic_string.h:3673:7: note: candidate: ‘std::basic_string<_CharT, _Traits, _Alloc>::basic_string(std::basic_string<_CharT, _Traits, _Alloc>::size_type, _CharT, const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>; std::basic_string<_CharT, _Traits, _Alloc>::size_type = long unsigned int]’
 3673 |       basic_string(size_type __n, _CharT __c, const _Alloc& __a = _Alloc())
      |       ^~~~~~~~~~~~
bbannier commented 1 day ago

I cannot reproduce this so some more details would be helpful. I checked the current development head ab69192894caef32e7bca316ec6fa5e65d4a40a4 against different versions of GCC. For that I modified an existing Debian-based setup to use gcc instead of the Debian images (the GCC Docker images are Debian-based),

-FROM debian:bookworm-slim
+FROM gcc:9

I also drop the installation of additional dependencies with pip.

This built fine for e.g., gcc-9.5 or gcc-11.2.0 (I could not find a gcc-11.2.1 release).

What did you do to see this error?

evantypanski commented 1 day ago

Does this happen to be ~on Windows or~ targeting Windows in some way? The suspicious part is SafeInt<uint64_t> converting into some standard size type there - GCC uses LLP64 in mingw, so that'd trigger the weirdness of 32 bit longs on Windows.

If this is using mingw, that's probably the problem. I think that'd be unsupported if it's true

EDIT: Regardless, you can see if it's the implicit conversion to the size type by using this patch, if you want to try something out. At least this way we avoid the implicit conversion from SafeInt to your ::size_type:

diff --git a/spicy/runtime/src/base64.cc b/spicy/runtime/src/base64.cc
index ab9559a7..14f1f411 100644
--- a/spicy/runtime/src/base64.cc
+++ b/spicy/runtime/src/base64.cc
@@ -35,7 +35,7 @@ hilti::rt::Bytes Stream::encode(const hilti::rt::Bytes& data) {
     if ( ! _state )
         throw Base64Error("encoding already finished");

-    std::string buf(data.size() * 2, {});
+    std::string buf(static_cast<long unsigned int>(data.size() * 2), {});
     auto len = base64_encode_block(data.data(), static_cast<int>(data.size()), buf.data(), &_state->estate);
     return hilti::rt::Bytes(buf.data(), len);
 }
@@ -47,7 +47,7 @@ hilti::rt::Bytes Stream::encode(const hilti::rt::stream::View& data) {
     hilti::rt::Bytes encoded;

     for ( auto block = data.firstBlock(); block; block = data.nextBlock(block) ) {
-        std::string buf(block->size * 2, {});
+        std::string buf(static_cast<long unsigned int>(block->size * 2), {});
         auto len = base64_encode_block(reinterpret_cast<const char*>(block->start), static_cast<int>(block->size),
                                        buf.data(), &_state->estate);
         encoded.append(hilti::rt::Bytes(buf.data(), len));
AmazingPP commented 1 day ago

I am using devtoolset-11-gcc on CentOS 7.

$ g++ -v
Using built-in specs.
COLLECT_GCC=g++
COLLECT_LTO_WRAPPER=/opt/rh/devtoolset-11/root/usr/libexec/gcc/x86_64-redhat-linux/11/lto-wrapper
Target: x86_64-redhat-linux
Configured with: ../configure --enable-bootstrap --enable-languages=c,c++,fortran,lto --prefix=/opt/rh/devtoolset-11/root/usr --mandir=/opt/rh/devtoolset-11/root/usr/share/man --infodir=/opt/rh/devtoolset-11/root/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --enable-multilib --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --with-linker-hash-style=gnu --with-default-libstdcxx-abi=gcc4-compatible --enable-plugin --enable-initfini-array --with-isl=/builddir/build/BUILD/gcc-11.2.1-20220127/obj-x86_64-redhat-linux/isl-install --enable-gnu-indirect-function --with-tune=generic --with-arch_32=x86-64 --build=x86_64-redhat-linux
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 11.2.1 20220127 (Red Hat 11.2.1-9) (GCC)
bbannier commented 1 day ago

Did you try whether above patch by @evantypanski helps in any way?

Otherwise could you share a Dockerfile which reproduces how you set up your system, and how you configure and build Spicy? We had a setup for building on centos-7 (with detoolset-9 at the time), but dropped support for it when centos-7 reached its end of life. Trying to resurrect that fails since mirrors went away.

IIRC devtoolset at least by default would inject a compiler, but e.g., still link against a system libstdc++ (so that binaries with devtoolset resolve when run in the host env). Not sure that could be the issue here.

AmazingPP commented 1 day ago

Did you try whether above patch by @evantypanski helps in any way?

I tried the patch, and it works perfectly.

Otherwise could you share a Dockerfile which reproduces how you set up your system, and how you configure and build Spicy?

No problem.

FROM centos:7

RUN sed -i \
    -e 's/^mirrorlist/#mirrorlist/' \
    -e 's/^#baseurl/baseurl/' \
    -e 's/mirror\.centos\.org/vault.centos.org/' \
    /etc/yum.repos.d/*.repo

RUN yum -y install \
    epel-release \
    centos-release-scl \
  && yum clean all && rm -rf /var/cache/yum

RUN sed -i \
    -e 's/^mirrorlist/#mirrorlist/' \
    -e 's/^#baseurl/baseurl/' \
    -e 's/^# baseurl/baseurl/' \
    -e 's/mirror\.centos\.org/vault.centos.org/' \
    /etc/yum.repos.d/CentOS-SCLo-scl*.repo

RUN yum -y install \
    bison \
    flex \
    cmake3 \
    make \
    which \
    git \
    zlib-devel \
    devtoolset-11 \
  && yum clean all && rm -rf /var/cache/yum

Then use the source /opt/rh/devtoolset-11/enable command to enable GCC 11 in bash. BTW, I didn't use any configure parameters, I just ./configure && make -C build.