[BUG] New compression code fails to build on armhf

[kiplingw]

Since enabling building with compression that was added in #1151 I noticed the builders are now choking on armhf (the old 32-bit ARM ISA). The compiler issues all appear to be related to the new compression code. This happens on all releases from Jammy to Noble.

Here is the salient portions of the compile log on Jammy:

In file included from ../include/pistache/http.h:36,
                 from ../src/common/http.cc:14:
../include/pistache/transport.h: In member function ‘size_t Pistache::Tcp::Transport::BufferHolder::offset() const’:
../include/pistache/transport.h:111:44: warning: conversion from ‘off_t’ {aka ‘long long int’} to ‘size_t’ {aka ‘unsigned int’} may change value [-Wconversion]
  111 |             size_t offset() const { return offset_; }
      |                                            ^~~~~~~
../src/common/http.cc: In member function ‘Pistache::Http::Private::State Pistache::Http::Private::BodyStep::parseContentLength(Pistache::StreamCursor&, const std::shared_ptr<Pistache::Http::Header::ContentLength>&)’:
../src/common/http.cc:431:56: warning: conversion from ‘long long unsigned int’ to ‘size_t’ {aka ‘unsigned int’} may change value [-Wconversion]
  431 |                 const size_t remaining = contentLength - bytesRead;
      |                                          ~~~~~~~~~~~~~~^~~~~~~~~~~
../src/common/http.cc:438:40: warning: conversion from ‘long long unsigned int’ to ‘std::__cxx11::basic_string<char>::size_type’ {aka ‘unsigned int’} may change value [-Wconversion]
  438 |                 message->body_.reserve(contentLength);
      |                                        ^~~~~~~~~~~~~
../src/common/http.cc:439:31: warning: conversion from ‘long long unsigned int’ to ‘size_t’ {aka ‘unsigned int’} may change value [-Wconversion]
  439 |                 if (!readBody(contentLength))
      |                               ^~~~~~~~~~~~~
../src/common/http.cc: In member function ‘Pistache::Async::Promise<int> Pistache::Http::ResponseWriter::sendImpl(Pistache::Http::Code, const char*, size_t, const Pistache::Http::Mime::MediaType&)’:
../src/common/http.cc:896:17: error: invalid conversion from ‘std::size_t*’ {aka ‘unsigned int*’} to ‘uLongf*’ {aka ‘long unsigned int*’} [-fpermissive]
  896 |                 &compressedSize,
      |                 ^~~~~~~~~~~~~~~
      |                 |
      |                 std::size_t* {aka unsigned int*}
In file included from /usr/include/zlib.h:34,
                 from ../include/pistache/http.h:39,
                 from ../src/common/http.cc:14:
/usr/include/zlib.h:1243:31: note:   initializing argument 2 of ‘int compress2(Bytef*, uLongf*, const Bytef*, uLong, int)’
 1243 | ZEXTERN int ZEXPORT compress2 OF((Bytef *dest,   uLongf *destLen,
      |                               ^~
../src/common/http.cc: In function ‘Pistache::Async::Promise<int> Pistache::Http::serveFile(Pistache::Http::ResponseWriter&, const string&, const Pistache::Http::Mime::MediaType&)’:
../src/common/http.cc:1124:31: warning: conversion from ‘__off64_t’ {aka ‘long long int’} to ‘size_t’ {aka ‘unsigned int’} may change value [-Wconversion]
 1124 |         const size_t len = sb.st_size;
      |                            ~~~^~~~~~~
[4/99] c++ -Isrc/libpistache.so.0.2.5.p -Isrc -I../src -Iinclude -I../include -I../subprojects/hinnant-date/include -fdiagnostics-color=always -DNDEBUG -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Wnon-virtual-dtor -Wextra -Wpedantic -std=c++17 -Wconversion -Wno-sign-conversion -Wno-missing-field-initializers -DPISTACHE_USE_SSL -DPISTACHE_USE_CONTENT_ENCODING_DEFLATE -g -O2 '-ffile-prefix-map=/<<PKGBUILDDIR>>=.' -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -pthread -MD -MQ src/libpistache.so.0.2.5.p/common_description.cc.o -MF src/libpistache.so.0.2.5.p/common_description.cc.o.d -o src/libpistache.so.0.2.5.p/common_description.cc.o -c ../src/common/description.cc
In file included from ../include/pistache/http.h:36,
                 from ../include/pistache/router.h:23,
                 from ../include/pistache/description.h:26,
                 from ../src/common/description.cc:14:
../include/pistache/transport.h: In member function ‘size_t Pistache::Tcp::Transport::BufferHolder::offset() const’:
../include/pistache/transport.h:111:44: warning: conversion from ‘off_t’ {aka ‘long long int’} to ‘size_t’ {aka ‘unsigned int’} may change value [-Wconversion]
  111 |             size_t offset() const { return offset_; }
      |                                            ^~~~~~~
[5/99] c++ -Isrc/libpistache.so.0.2.5.p -Isrc -I../src -Iinclude -I../include -I../subprojects/hinnant-date/include -fdiagnostics-color=always -DNDEBUG -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Wnon-virtual-dtor -Wextra -Wpedantic -std=c++17 -Wconversion -Wno-sign-conversion -Wno-missing-field-initializers -DPISTACHE_USE_SSL -DPISTACHE_USE_CONTENT_ENCODING_DEFLATE -g -O2 '-ffile-prefix-map=/<<PKGBUILDDIR>>=.' -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -pthread -MD -MQ src/libpistache.so.0.2.5.p/common_http_header.cc.o -MF src/libpistache.so.0.2.5.p/common_http_header.cc.o.d -o src/libpistache.so.0.2.5.p/common_http_header.cc.o -c ../src/common/http_header.cc
In file included from ../include/pistache/http.h:36,
                 from ../src/common/http_header.cc:16:
../include/pistache/transport.h: In member function ‘size_t Pistache::Tcp::Transport::BufferHolder::offset() const’:
../include/pistache/transport.h:111:44: warning: conversion from ‘off_t’ {aka ‘long long int’} to ‘size_t’ {aka ‘unsigned int’} may change value [-Wconversion]
  111 |             size_t offset() const { return offset_; }
      |                                            ^~~~~~~
In file included from /usr/include/c++/11/vector:72,
                 from ../include/pistache/base64.h:19,
                 from ../src/common/http_header.cc:13:
/usr/include/c++/11/bits/vector.tcc: In member function ‘void std::vector<_Tp, _Alloc>::_M_realloc_insert(std::vector<_Tp, _Alloc>::iterator, _Args&& ...) [with _Args = {const Pistache::Http::CacheDirective&}; _Tp = Pistache::Http::CacheDirective; _Alloc = std::allocator<Pistache::Http::CacheDirective>]’:
/usr/include/c++/11/bits/vector.tcc:426:7: note: parameter passing for argument of type ‘std::vector<Pistache::Http::CacheDirective>::iterator’ changed in GCC 7.1
  426 |       vector<_Tp, _Alloc>::
      |       ^~~~~~~~~~~~~~~~~~~
In file included from /usr/include/c++/11/vector:67,
                 from ../include/pistache/base64.h:19,
                 from ../src/common/http_header.cc:13:
/usr/include/c++/11/bits/stl_vector.h: In constructor ‘Pistache::Http::Header::CacheControl::CacheControl(Pistache::Http::CacheDirective)’:
/usr/include/c++/11/bits/stl_vector.h:1198:28: note: parameter passing for argument of type ‘__gnu_cxx::__normal_iterator<Pistache::Http::CacheDirective*, std::vector<Pistache::Http::CacheDirective> >’ changed in GCC 7.1
 1198 |           _M_realloc_insert(end(), __x);
      |           ~~~~~~~~~~~~~~~~~^~~~~~~~~~~~
/usr/include/c++/11/bits/stl_vector.h: In member function ‘void Pistache::Http::Header::CacheControl::addDirectives(const std::vector<Pistache::Http::CacheDirective>&)’:
/usr/include/c++/11/bits/stl_vector.h:1198:28: note: parameter passing for argument of type ‘__gnu_cxx::__normal_iterator<Pistache::Http::CacheDirective*, std::vector<Pistache::Http::CacheDirective> >’ changed in GCC 7.1
 1198 |           _M_realloc_insert(end(), __x);
      |           ~~~~~~~~~~~~~~~~~^~~~~~~~~~~~
/usr/include/c++/11/bits/stl_vector.h: In member function ‘void Pistache::Http::Header::CacheControl::addDirective(Pistache::Http::CacheDirective)’:
/usr/include/c++/11/bits/stl_vector.h:1198:28: note: parameter passing for argument of type ‘__gnu_cxx::__normal_iterator<Pistache::Http::CacheDirective*, std::vector<Pistache::Http::CacheDirective> >’ changed in GCC 7.1
 1198 |           _M_realloc_insert(end(), __x);
      |           ~~~~~~~~~~~~~~~~~^~~~~~~~~~~~
In file included from /usr/include/c++/11/vector:72,
                 from ../include/pistache/base64.h:19,
                 from ../src/common/http_header.cc:13:
/usr/include/c++/11/bits/vector.tcc: In member function ‘void std::vector<_Tp, _Alloc>::_M_realloc_insert(std::vector<_Tp, _Alloc>::iterator, _Args&& ...) [with _Args = {const Pistache::Http::CacheDirective::Directive&}; _Tp = Pistache::Http::CacheDirective; _Alloc = std::allocator<Pistache::Http::CacheDirective>]’:
/usr/include/c++/11/bits/vector.tcc:426:7: note: parameter passing for argument of type ‘std::vector<Pistache::Http::CacheDirective>::iterator’ changed in GCC 7.1
  426 |       vector<_Tp, _Alloc>::
      |       ^~~~~~~~~~~~~~~~~~~
/usr/include/c++/11/bits/vector.tcc: In member function ‘void std::vector<_Tp, _Alloc>::_M_realloc_insert(std::vector<_Tp, _Alloc>::iterator, _Args&& ...) [with _Args = {const Pistache::Http::CacheDirective::Directive&, std::chrono::duration<long long int, std::ratio<1, 1> >}; _Tp = Pistache::Http::CacheDirective; _Alloc = std::allocator<Pistache::Http::CacheDirective>]’:
/usr/include/c++/11/bits/vector.tcc:426:7: note: parameter passing for argument of type ‘std::vector<Pistache::Http::CacheDirective>::iterator’ changed in GCC 7.1
/usr/include/c++/11/bits/vector.tcc: In member function ‘virtual void Pistache::Http::Header::CacheControl::parseRaw(const char*, size_t)’:
/usr/include/c++/11/bits/vector.tcc:121:28: note: parameter passing for argument of type ‘__gnu_cxx::__normal_iterator<Pistache::Http::CacheDirective*, std::vector<Pistache::Http::CacheDirective> >’ changed in GCC 7.1
  121 |           _M_realloc_insert(end(), std::forward<_Args>(__args)...);
      |           ~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/c++/11/bits/vector.tcc:121:28: note: parameter passing for argument of type ‘__gnu_cxx::__normal_iterator<Pistache::Http::CacheDirective*, std::vector<Pistache::Http::CacheDirective> >’ changed in GCC 7.1
  121 |           _M_realloc_insert(end(), std::forward<_Args>(__args)...);
      |           ~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[6/99] c++ -Isrc/libpistache.so.0.2.5.p -Isrc -I../src -Iinclude -I../include -I../subprojects/hinnant-date/include -fdiagnostics-color=always -DNDEBUG -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Wnon-virtual-dtor -Wextra -Wpedantic -std=c++17 -Wconversion -Wno-sign-conversion -Wno-missing-field-initializers -DPISTACHE_USE_SSL -DPISTACHE_USE_CONTENT_ENCODING_DEFLATE -g -O2 '-ffile-prefix-map=/<<PKGBUILDDIR>>=.' -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -pthread -MD -MQ src/libpistache.so.0.2.5.p/common_http_defs.cc.o -MF src/libpistache.so.0.2.5.p/common_http_defs.cc.o.d -o src/libpistache.so.0.2.5.p/common_http_defs.cc.o -c ../src/common/http_defs.cc
ninja: build stopped: subcommand failed.

[Tachi107]

Try changing this variable to zlib's type uLongf:

https://github.com/pistacheio/pistache/blob/e65f05aca53357fea500d4550d659051b6ae6444/src/common/http.cc#L886-L888

[kiplingw]

I'll give it a try. It's a bit complicated to get a test bed setup for armhf to test fix.

[Tachi107]

Il giorno mar 28 nov 2023 alle 15:01:07 -08:00:00, Kip @.***> ha scritto:

I'll give it a try. It's a bit complicated to get a test bed setup for armhf to test fix.

autopkgtest-virt-qemu can be your friend :)

[kiplingw]

Yeah that's what I'm going to try, or with LXC.

[kiplingw]

autopkgtest-virt-qemu(1) I couldn't get to work. But I was able to replicate the issue and a fix with the help of godbolt's compiler explorer. Pushing fix now.