search

fluent / fluent-bit

Fast and Lightweight Logs and Metrics processor for Linux, BSD, OSX and Windows
https://fluentbit.io
Apache License 2.0
5.86k stars 1.58k forks source link

Fail to build docker image with avro encoder in output_kafka plugin #7180

Closed shuishou1983 closed 1 year ago

shuishou1983 commented 1 year ago

Bug Report

Describe the bug Fail to build docker image if set -DFLB_AVRO_ENCODER=On

To Reproduce

Expected behavior The command run without error and generate the docker image, but it got following error. 



**Your Environment**
<!--- Include as many relevant details about the environment you experienced the bug in -->
* Version used: 2.1.0
* Configuration: 
* Environment name and version (e.g. Kubernetes? What version?): Windows and Ubuntu
* Server type and version:
* Operating System and version:
* Filters and plugins: output_kafka
patrick-stephens commented 1 year ago

Possibly a duplicate of https://github.com/fluent/fluent-bit/issues/7175 and therefore resolved by https://github.com/fluent/fluent-bit/pull/7176

Could you try the changes on https://github.com/fluent/fluent-bit/pull/7176 to see if they resolve it @shuishou1983 ?

shuishou1983 commented 1 year ago

Thank you for the update, @patrick-stephens . Had a try but still got the same error. Here is the detail output: [+] Building 32.8s (24/32) => [internal] load build definition from Dockerfile 0.1s => => transferring dockerfile: 6.22kB 0.0s => [internal] load .dockerignore 0.1s => => transferring context: 34B 0.0s => [internal] load metadata for gcr.io/distroless/cc-debian11:latest 0.4s => [internal] load metadata for docker.io/multiarch/qemu-user-static:x86_64-aarch64 2.1s => [internal] load metadata for docker.io/multiarch/qemu-user-static:x86_64-arm 2.1s => [internal] load metadata for docker.io/library/debian:bullseye-slim 1.9s => [auth] library/debian:pull token for registry-1.docker.io 0.0s => [auth] multiarch/qemu-user-static:pull token for registry-1.docker.io 0.0s => [qemu-arm64 1/1] FROM docker.io/multiarch/qemu-user-static:x86_64-aarch64@sha256:6218d0b52fa96ad8fd4ea5596c9cb448915641c664ef1bb8d00c659a4fb748b5 0.0s => [production 1/4] FROM gcr.io/distroless/cc-debian11@sha256:8aad707f96620ee89e27febef51b01c6ff244277a3560fcfcfbe68633ef09193 0.0s => [internal] load build context 1.4s => => transferring context: 580.05kB 1.3s => [deb-extractor 1/6] FROM docker.io/library/debian:bullseye-slim@sha256:9404b05bd09b57c76eccc0c5505b3c88b5feccac808d9b193a4fbac87bb44745 0.0s => [qemu-arm32 1/1] FROM docker.io/multiarch/qemu-user-static:x86_64-arm@sha256:339b9e5822de7ed6412c985fee69d578af03262e223971e6fe0adaa31386df31 0.0s => CACHED [deb-extractor 2/6] COPY --from=qemu-arm32 /usr/bin/qemu-arm-static /usr/bin/ 0.0s => CACHED [deb-extractor 3/6] COPY --from=qemu-arm64 /usr/bin/qemu-aarch64-static /usr/bin/ 0.0s => CACHED [deb-extractor 4/6] WORKDIR /tmp 0.0s => CANCELED [deb-extractor 5/6] RUN echo "deb http://deb.debian.org/debian bullseye-backports main" >> /etc/apt/sources.list && apt-get clean && apt-get update && 30.3s => CACHED [builder-base 4/8] RUN mkdir -p /fluent-bit/bin /fluent-bit/etc /fluent-bit/log /src/fluent-bit 0.0s => CACHED [builder-base 5/8] RUN echo "deb http://deb.debian.org/debian bullseye-backports main" >> /etc/apt/sources.list && apt-get clean && apt-get update && apt- 0.0s => CACHED [builder-base 6/8] WORKDIR /src/fluent-bit/ 0.0s => [builder-base 7/8] COPY . ./ 14.7s => [builder-base 8/8] RUN ls -al 0.7s => [builder 1/6] WORKDIR /src/fluent-bit/build/ 0.4s => ERROR [builder 2/6] RUN cmake -DFLB_RELEASE=On -DFLB_OUT_KAFKA=On -DFLB_TLS=On -DFLB_HTTP_SERVER=On -DFLB_AVRO_ENCODER=On .. 12.7s

[builder 2/6] RUN cmake -DFLB_RELEASE=On -DFLB_OUT_KAFKA=On -DFLB_TLS=On -DFLB_HTTP_SERVER=On -DFLB_AVRO_ENCODER=On ..:

26 0.789 -- The C compiler identification is GNU 10.2.1

26 0.811 -- Detecting C compiler ABI info

26 0.984 -- Detecting C compiler ABI info - done

26 1.007 -- Check for working C compiler: /usr/bin/cc - skipped

26 1.008 -- Detecting C compile features

26 1.011 -- Detecting C compile features - done

26 1.086 -- Found PkgConfig: /usr/bin/pkg-config (found version "0.29.2")

26 1.129 Package systemd was not found in the pkg-config search path.

26 1.129 Perhaps you should add the directory containing `systemd.pc'

26 1.129 to the PKG_CONFIG_PATH environment variable

26 1.129 No package 'systemd' found

26 1.134 -- Found Journald: /usr/lib/x86_64-linux-gnu/libsystemd.so

26 1.137 -- Found Monkey: /src/fluent-bit/lib/monkey/include

26 1.167 -- Could NOT find LibEdit (missing: libedit_INCLUDE_DIRS libedit_LIBRARIES)

26 1.181 -- Found Git: /usr/bin/git (found version "2.30.2")

26 1.228 -- Git hash: 62fda8dace967894792370325e33310870e2a75c

26 1.254 -- Found FLEX: /usr/bin/flex (found suitable version "2.6.4", minimum required is "2")

26 1.271 -- Found BISON: /usr/bin/bison (found suitable version "3.7.5", minimum required is "3")

26 1.996 WarningIPO is not supported on this platform

26 1.997 -- Enabling FLB_REGEX since FLB_PARSER requires

26 2.010 -- Performing Test CFL_HAVE_TIMESPEC_GET

26 2.427 -- Performing Test CFL_HAVE_TIMESPEC_GET - Success

26 2.428 -- Performing Test CFL_HAVE_GMTIME_R

26 2.681 -- Performing Test CFL_HAVE_GMTIME_R - Success

26 2.681 -- Performing Test CFL_HAVE_GMTIME_S

26 2.986 -- Performing Test CFL_HAVE_GMTIME_S - Failed

26 2.986 -- Performing Test CFL_HAVE_CLOCK_GET_TIME

26 3.108 -- Performing Test CFL_HAVE_CLOCK_GET_TIME - Failed

26 3.161 -- xxHash build type: RelWithDebInfo

26 3.661 -- The CXX compiler identification is GNU 10.2.1

26 3.688 -- Detecting CXX compiler ABI info

26 3.816 -- Detecting CXX compiler ABI info - done

26 3.844 -- Check for working CXX compiler: /usr/bin/c++ - skipped

26 3.845 -- Detecting CXX compile features

26 3.847 -- Detecting CXX compile features - done

26 3.850 -- Check if the system is big endian

26 3.850 -- Searching 16 bit integer

26 3.851 -- Looking for sys/types.h

26 3.963 -- Looking for sys/types.h - found

26 3.963 -- Looking for stdint.h

26 4.088 -- Looking for stdint.h - found

26 4.100 -- Looking for stddef.h

26 4.191 -- Looking for stddef.h - found

26 4.192 -- Check size of unsigned short

26 4.296 -- Check size of unsigned short - done

26 4.296 -- Searching 16 bit integer - Using unsigned short

26 4.446 -- Check if the system is big endian - little endian

26 4.490 -- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY

26 4.726 -- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY - Success

26 4.728 -- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY

26 4.967 -- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY - Success

26 4.968 -- Performing Test COMPILER_HAS_DEPRECATED_ATTR

26 5.237 -- Performing Test COMPILER_HAS_DEPRECATED_ATTR - Success

26 5.273 Entering /src/fluent-bit/lib/lwrb/lwrb/CMakeLists.txt

26 5.273 Exiting /src/fluent-bit/lib/lwrb/lwrb/CMakeLists.txt

26 5.274 -- Performing Test HAS_NO_FORMAT_TRUNCATION

26 5.384 C compiler: GNU

26 5.385 -- Performing Test HAS_NO_FORMAT_TRUNCATION - Success

26 5.385 -- Looking for include file endian.h

26 5.483 -- Looking for include file endian.h - found

26 5.488 -- Looking for include file fcntl.h

26 5.604 -- Looking for include file fcntl.h - found

26 5.605 -- Looking for include file sched.h

26 5.713 -- Looking for include file sched.h - found

26 5.714 -- Looking for include file unistd.h

26 5.823 -- Looking for include file unistd.h - found

26 5.826 -- Looking for include file sys/param.h

26 5.982 -- Looking for include file sys/param.h - found

26 5.983 -- Looking for include file sys/stat.h

26 6.123 -- Looking for include file sys/stat.h - found

26 6.127 -- Looking for include file sys/time.h

26 6.304 -- Looking for include file sys/time.h - found

26 6.304 -- Looking for close

26 6.501 -- Looking for close - found

26 6.503 -- Looking for getpid

26 6.601 -- Looking for getpid - found

26 6.603 -- Looking for gettimeofday

26 6.717 -- Looking for gettimeofday - found

26 6.720 -- Looking for open

26 6.833 -- Looking for open - found

26 6.833 -- Looking for read

26 6.973 -- Looking for read - found

26 6.974 -- Looking for sched_yield

26 7.098 -- Looking for sched_yield - found

26 7.101 -- Check size of __int64

26 7.150 -- Check size of __int64 - failed

26 7.152 -- Check size of int64_t

26 7.260 -- Check size of int64_t - done

26 7.261 -- Check size of long long

26 7.354 -- Check size of long long - done

26 7.355 -- Check size of int32_t

26 7.567 -- Check size of int32_t - done

26 7.569 -- Check size of __int32

26 7.678 -- Check size of __int32 - failed

26 7.680 -- Check size of long

26 7.788 -- Check size of long - done

26 7.792 -- Check size of int

26 7.946 -- Check size of int - done

26 7.947 -- Check size of unsigned long

26 8.070 -- Check size of unsigned long - done

26 8.074 -- Check size of unsigned int

26 8.204 -- Check size of unsigned int - done

26 8.207 -- Check size of unsigned short

26 8.349 -- Check size of unsigned short - done

26 8.351 -- Check size of uint32_t

26 8.465 -- Check size of uint32_t - done

26 8.467 -- Check size of __uint32

26 8.520 -- Check size of __uint32 - failed

26 8.525 -- Check size of uint16_t

26 8.653 -- Check size of uint16_t - done

26 8.658 -- Check size of __uint16

26 8.728 -- Check size of __uint16 - failed

26 8.732 -- Check size of uint8_t

26 8.849 -- Check size of uint8_t - done

26 8.852 -- Check size of __uint8

26 8.911 -- Check size of __uint8 - failed

26 8.912 -- Check size of ssize_t

26 9.077 -- Check size of ssize_t - done

26 9.079 -- Check size of SSIZE_T

26 9.201 -- Check size of SSIZE_T - failed

26 9.203 -- Looking for strtoll

26 9.405 -- Looking for strtoll - found

26 9.406 -- Looking for strtoq

26 9.573 -- Looking for strtoq - found

26 9.578 -- Looking for _strtoi64

26 9.786 -- Looking for _strtoi64 - not found

26 9.786 -- Looking for include file locale.h

26 10.02 -- Looking for include file locale.h - found

26 10.04 -- Looking for localeconv

26 10.29 -- Looking for localeconv - found

26 10.29 -- Looking for setlocale

26 10.62 -- Looking for setlocale - found

26 10.62 -- Performing Test HAVE_INLINE

26 10.87 -- Performing Test HAVE_INLINE - Success

26 10.87 -- Performing Test HAVE___INLINE

26 11.11 -- Performing Test HAVE___INLINE - Success

26 11.11 -- Performing Test HAVE_INLINE

26 11.36 -- Performing Test HAVE_INLINE - Success

26 11.36 -- Performing Test HAVE_SYNC_BUILTINS

26 11.61 -- Performing Test HAVE_SYNC_BUILTINS - Success

26 11.61 -- Performing Test HAVE_ATOMIC_BUILTINS

26 11.96 -- Performing Test HAVE_ATOMIC_BUILTINS - Success

26 12.06 CMake Error at lib/avro/CMakeLists.txt:39 (message):

26 12.06 Cannot determine Avro version number

26 12.06

26 12.06

26 12.07 -- Configuring incomplete, errors occurred!

26 12.07 See also "/src/fluent-bit/build/CMakeFiles/CMakeOutput.log".

26 12.07 See also "/src/fluent-bit/build/CMakeFiles/CMakeError.log".

executor failed running [/bin/sh -c cmake -DFLB_RELEASE=On -DFLB_OUT_KAFKA=On -DFLB_TLS=On -DFLB_HTTP_SERVER=On -DFLB_AVRO_ENCODER=On ..]: exit code: 1

patrick-stephens commented 1 year ago

From what I can tell, that error is down to an issue running the version.sh script in libavro: https://github.com/apache/avro/blob/0c53710f511e36340d73c3f96aef02771d886156/lang/c/CMakeLists.txt#L39

I cannot say why but might be worth looking at the requirements around it and also things like are you running on a partition with noexec or something like that.

I also see some use of relative paths so hopefully that is not a problem: https://github.com/apache/avro/blob/0c53710f511e36340d73c3f96aef02771d886156/lang/c/version.sh#L64

github-actions[bot] commented 1 year ago

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days. Maintainers can add the exempt-stale label.

github-actions[bot] commented 1 year ago

This issue was closed because it has been stalled for 5 days with no activity.