search

gearman / gearmand

http://gearman.org/
Other
740 stars 137 forks source link

Build in Alpine #293

Closed gitHusband closed 4 years ago

gitHusband commented 4 years ago

Hi!

I got some problem when I tried to build gearmand in Docker-Alpine(gcc v9.3.0).

As you know, when build gearmand with version 1.1.91.1 and below, there has a bug which was fixed in #289. error: 'environ' was declared 'extern' and later 'static' [-fpermissive]

So I try to build master.

There are two bugs for now:

  1. git clone https://github.com/gearman/gearmand && cd gearmand && ./bootstrap.sh -a,
    got an error: ./bootstrap.sh:317: An attempt was made to set an invalid VENDOR=alpine You may need to fix this bug. But I can do these things using CentOS and then clone the codes into Docker-Alpine, so I can make it in Alpine.
  2. ./configure -- done! make - Bug occurred: 
      CXX      libhashkit/libhashkit_libhashkitinc_la-function.lo
**libhashkit/fnv_64.cc:41:5: error: "__WORDSIZE" is not defined, evaluates to 0 [-Werror=undef]**
41 | #if __WORDSIZE == 64 && defined(HAVE_FNV64_HASH)
  |     ^~~~~~~~~~
CXX      libhashkit/libhashkit_libhashkitinc_la-has.lo
cc1plus: all warnings being treated as errors
make[2]: *** [Makefile:6803: libhashkit/libhashkit_libhashkitinc_la-fnv_64.lo] Error 1
make[2]: *** Waiting for unfinished jobs....
libhashkit/has.cc:47:5: error: "__WORDSIZE" is not defined, evaluates to 0 [-Werror=undef]
47 | #if __WORDSIZE == 64 && defined(HAVE_FNV64_HASH)
  |     ^~~~~~~~~~
cc1plus: all warnings being treated as errors
make[2]: *** [Makefile:6817: libhashkit/libhashkit_libhashkitinc_la-has.lo] Error 1
make[2]: Leaving directory '/tmp/gearmand-master-configure'
make[1]: *** [Makefile:8307: all-recursive] Error 1
make[1]: Leaving directory '/tmp/gearmand-master-configure'
make: *** [Makefile:3069: all] Error 2

    Can you please help to fix these bugs? Thanks!

gitHusband commented 4 years ago

I fix libhashkit/common.h in my computer and then I can make successfully.

  #ifndef __WORDSIZE                
  # ifdef __MINGW32__               
  #  define __WORDSIZE 32           
+ # else                            
+ #  define __WORDSIZE 64    
  # endif                           
  #endif
p-alik commented 4 years ago

Thank you, @gitHusband! Patched master could be faultless build on Ubuntu 18.04 with gcc 7.5.0.

gitHusband commented 4 years ago

Thank you, @gitHusband! Patched master could be faultless build on Ubuntu 18.04 with gcc 7.5.0.

It's my pleasure. Would you fix the two bugs when you have time? @p-alik

p-alik commented 4 years ago

Yes, I'll do that.

gitHusband commented 4 years ago

Cool! Thanks!

esabol commented 4 years ago

Using a modified bootstrap.sh and libhashkit/common.h, I was able to bootstrap, build, and test. Here's the Dockerfile I used:

FROM alpine
MAINTAINER gearmand

# Install packages
RUN apk add musl-dev gcc g++ autoconf automake m4 git libtool make bash py3-sphinx \
            file util-linux-dev libuuid libevent-dev gperf boost-dev openssl-dev

# Switch to non-root user
RUN adduser --disabled-password --shell /bin/bash gearman
USER gearman

ARG GEARMAN_REPO=https://github.com/gearman/gearmand
RUN cd /tmp && git clone --depth 1 --branch master ${GEARMAN_REPO}.git
WORKDIR /tmp/gearmand
COPY bootstrap.sh /tmp/gearmand/
COPY common.h /tmp/gearmand/libhashkit/
RUN ./bootstrap.sh -a
RUN ./configure --enable-ssl 2>&1 | tee ./configure.log
RUN make 2>&1 | tee ./build.log
RUN make test 2>&1 | tee ./test.log

Here are the diffs on my bootstrap.sh just for comparison:

--- /path/to/github/gearmand/bootstrap.sh    2019-11-27 17:02:43.416800000 -0500
+++ ./bootstrap.sh      2020-07-30 18:51:38.209550000 -0400
@@ -194,6 +194,9 @@
     opensuse*)
       VENDOR_DISTRIBUTION='opensuse'
       ;;
+    alpine)
+      VENDOR_DISTRIBUTION='alpine'
+      ;;
     *)
       die "attempt to set an invalid VENDOR_DISTRIBUTION=$dist"
       ;;
@@ -263,6 +266,9 @@
     opensuse)
       VENDOR_RELEASE="$release"
       ;;
+    alpine)
+      VENDOR_RELEASE="$release"
+      ;;
     unknown)
       die "attempt to set VENDOR_RELEASE without setting VENDOR_DISTRIBUTION"
       ;;
@@ -273,13 +279,16 @@
 }

-#  Valid values are: apple, redhat, centos, canonical, oracle, suse
+#  Valid values are: apple, redhat, centos, canonical, oracle, suse, alpine
 set_VENDOR ()
 {
   local vendor
   vendor="$(echo "$1" | tr '[:upper:]' '[:lower:]')"

   case $vendor in
+    alpine)
+      VENDOR='alpine'
+      ;;
     apple)
       VENDOR='apple'
       ;;
@@ -372,6 +381,10 @@
     # shellcheck disable=SC1091
     source '/etc/lsb-release'
     set_VENDOR 'canonical' "$DISTRIB_ID" "$DISTRIB_CODENAME"
+  elif [[ -f '/etc/alpine-release' ]]; then
+    local alpine_version
+    alpine_version="$(cat /etc/alpine-release)"
+    set_VENDOR 'alpine' 'alpine' "$alpine_version"
   fi

   rebuild_host_os

Hope this helps!

p-alik commented 4 years ago

@esabol, could you put the snippets in a PR, please? Regarding libhashkit, it's a part of geamand and libmemcached. geamand version of the lib is a bit elder. For instance update_continuum seems to be useless

08:34 $ git grep update_continuum
libhashkit/common.h:/* int update_continuum(hashkit_st *hashkit); */
libhashkit/ketama.cc:int update_continuum(hashkit_st *hashkit)

Removing update_continuum from libhashkit/common.h appears to be harmless. The remaining md5_signature seems to be alike libhashkit_md5_signature

git grep "md5_signature(const"
libhashkit-1.0/algorithm.h:void libhashkit_md5_signature(const unsigned char *key, size_t length, unsigned char *result);
libhashkit/algorithm.cc:void libhashkit_md5_signature(const unsigned char *key, size_t length, unsigned char *result)
libhashkit/common.h:void md5_signature(const unsigned char *key, unsigned int length, unsigned char *result);
libhashkit/md5.cc:void md5_signature(const unsigned char *key, unsigned int length, unsigned char *result)

Wouldn't replacement of md5_signature by libhashkit_md5_signature make libhashkit/common.h obsolete?

esabol commented 4 years ago

Wouldn't replacement of md5_signature by libhashkit_md5_signature make libhashkit/common.h obsolete?

I think libhashkit has been copied here from another project (libmemcached?). If so, I think it makes sense to just keep it as it is in order to facilitate comparisons with the upstream codebase.

p-alik commented 4 years ago

I think libhashkit has been copied here from another project (libmemcached?)

@dormando, @BrianAker, could you share your knowledge regarding libhashkit. Would you recommend to keep it in gearmand code base?

If so, I think it makes sense to just keep it as it is in order to facilitate comparisons with the upstream codebase.

I'm of the opposite opinion because:

esabol commented 4 years ago

@p-alik: Looking through the code, it appears to me that libhashkit_md5_signature is the C++ interface and md5_signature is the C interface. Am I understanding that correctly?

Are you also saying libhashkit_md5_signature is the only routine used by gearmand? And the memcached plugin doesn't use it either? If so, you make a convincing case.

p-alik commented 4 years ago

@p-alik: Looking through the code, it appears to me that libhashkit_md5_signature is the C++ interface and md5_signature is the C interface. Am I understanding that correctly?

libhashkit_md5_signature and md5_signature are mostly equal.

Yeah, I saw that, but one is in an extern "C" block and the other one isn't. I suspect one is the public (external) interface and the other is the private (internal) implementation of the public interface. In other words, I do not think anyone is supposed to use md5_signature and only libhashkit_md5_signature is supposed to be used. (I don't actually know this. I'm just inferring.)

Are you also saying libhashkit_md5_signature is the only routine used by gearmand? And the memcached plugin doesn't use it either?

It seems there is very limited demand for libhashkit at all. But I'll check it accurately.

I see it's used by libgearman. If you eliminate it, that will change the symbols in libgearman.so, I think? And that will require bumping version numbers on libgearman.so, right?

SpamapS commented 4 years ago

Can you please confirm whether or not master builds for you on Alpine now?

p-alik commented 4 years ago

Can you please confirm whether or not master builds for you on Alpine now?

295 provided successful build on Alpine

esabol commented 4 years ago

My Docker image of the master branch builds and tests fine:

FROM alpine

MAINTAINER gearmand

# Install packages
RUN apk add --no-cache \
            musl-dev gcc g++ autoconf automake m4 git libtool make bash file \
            py3-sphinx util-linux-dev libuuid libevent-dev gperf boost-dev \
            openssl-dev

# Switch to non-root user
RUN adduser --disabled-password --shell /bin/bash gearman
USER gearman

ARG GEARMAN_REPO=https://github.com/gearman/gearmand

RUN cd /tmp && git clone --depth 1 --branch master ${GEARMAN_REPO}.git
WORKDIR /tmp/gearmand
RUN ./bootstrap.sh -a
RUN ./configure --enable-ssl 2>&1 | tee ./configure.log
RUN make 2>&1 | tee ./build.log
RUN make test 2>&1 | tee ./test.log
============================================================================
Testsuite summary for gearmand 67407a4
============================================================================
# TOTAL: 35
# PASS:  26
# SKIP:  7
# XFAIL: 2
# FAIL:  0
# XPASS: 0
# ERROR: 0
============================================================================
gitHusband commented 4 years ago

Cool, I can build Docker Alpine image of the master branch successfully now. Thanks! @SpamapS @p-alik @esabol