search

fluent / fluent-bit

Fast and Lightweight Logs and Metrics processor for Linux, BSD, OSX and Windows
https://fluentbit.io
Apache License 2.0
5.55k stars 1.52k forks source link

fluent-bit does not compile with with only specific options enabled #6464

Closed nickbroon closed 1 year ago

nickbroon commented 1 year ago

Bug Report

Describe the bug

fluent-bit does not compile when only these options are enabled and all others are disabled:

    -DFLB_REGEX=ON \
    -DFLB_UTF8_ENCODER=ON \
    -DFLB_PARSER=ON \
    -DFLB_TLS=ON \
    -DFLB_BINARY=ON \
    -DFLB_SHARED_LIB=ON \
    -DFLB_INOTIFY=ON \
    -DFLB_BACKTRACE=ON \
    -DFLB_RECORD_ACCESSOR=ON \
    -DFLB_CONFIG_YAML=ON \
    -DFLB_IN_SYSTEMD=ON \
    -DFLB_OUT_FORWARD=ON \
    -DFLB_FILTER_MODIFY=ON \
    -DFLB_FILTER_RECORD_MODIFIER=ON \

I'm guessing there is an option thats not really optional, or an option that's depended on by some other option.

$ cmake \ -DFLB_REGEX=ON \ -DFLB_UTF8_ENCODER=ON \ -DFLB_PARSER=ON \ -DFLB_TLS=ON \ -DFLB_BINARY=ON \ -DFLB_SHARED_LIB=ON \ -DFLB_INOTIFY=ON \ -DFLB_BACKTRACE=ON \ -DFLB_RECORD_ACCESSOR=ON \ -DFLB_CONFIG_YAML=ON \ -DFLB_IN_SYSTEMD=ON \ -DFLB_OUT_FORWARD=ON \ -DFLB_FILTER_MODIFY=ON \ -DFLB_FILTER_RECORD_MODIFIER=ON \ -DFLB_ALL=OFF \ -DFLB_DEBUG=OFF \ -DFLB_RELEASE=OFF \ -DFLB_SMALL=OFF \ -DFLB_COVERAGE=OFF \ -DFLB_JEMALLOC=OFF \ -DFLB_EXAMPLES=OFF \ -DFLB_VALGRIND=OFF \ -DFLB_TRACE=OFF \ -DFLB_CHUNK_TRACE=OFF \ -DFLB_TESTS_RUNTIME=OFF \ -DFLB_TESTS_INTERNAL=OFF \ -DFLB_TESTS_INTERNAL_FUZZ=OFF \ -DFLB_TESTS_OSSFUZZ=OFF \ -DFLB_MTRACE=OFF \ -DFLB_POSIX_TLS=OFF \ -DFLB_SQLDB=OFF \ -DFLB_HTTP_SERVER=OFF \ -DFLB_LUAJIT=OFF \ -DFLB_SIGNV4=OFF \ -DFLB_AWS=OFF \ -DFLB_STATIC_CONF=OFF \ -DFLB_STREAM_PROCESSOR=OFF \ -DFLB_CORO_STACK_SIZE=OFF \ -DFLB_AVRO_ENCODER=OFF \ -DFLB_AWS_ERROR_REPORTER=OFF \ -DFLB_ARROW=OFF \ -DFLB_WINDOWS_DEFAULTS=OFF \ -DFLB_WASM=OFF \ -DFLB_WAMRC=OFF \ -DFLB_METRICS=OFF \ -DFLB_PROXY_GO=OFF \ -DFLB_CUSTOM_CALYPTIA=OFF \ -DFLB_IN_CPU=OFF \ -DFLB_IN_THERMAL=OFF \ -DFLB_IN_DISK=OFF \ -DFLB_IN_DOCKER=OFF \ -DFLB_IN_DOCKER_EVENTS=OFF \ -DFLB_IN_EXEC=OFF \ -DFLB_IN_EXEC_WASI=OFF \ -DFLB_IN_EVENT_TEST=OFF \ -DFLB_IN_EVENT_TYPE=OFF \ -DFLB_IN_FLUENTBIT_METRICS=OFF \ -DFLB_IN_FORWARD=OFF \ -DFLB_IN_HEALTH=OFF \ -DFLB_IN_HTTP=OFF \ -DFLB_IN_MEM=OFF \ -DFLB_IN_KAFKA=OFF \ -DFLB_IN_KMSG=OFF \ -DFLB_IN_LIB=OFF \ -DFLB_IN_RANDOM=OFF \ -DFLB_IN_SERIAL=OFF \ -DFLB_IN_STDIN=OFF \ -DFLB_IN_SYSLOG=OFF \ -DFLB_IN_TAIL=OFF \ -DFLB_IN_UDP=OFF \ -DFLB_IN_TCP=OFF \ -DFLB_IN_UNIX_SOCKET=OFF \ -DFLB_IN_MQTT=OFF \ -DFLB_IN_HEAD=OFF \ -DFLB_IN_PROC=OFF \ -DFLB_IN_DUMMY=OFF \ -DFLB_IN_NGINX_EXPORTER_METRICS=OFF \ -DFLB_IN_NETIF=OFF \ -DFLB_IN_WINLOG=OFF \ -DFLB_IN_WINSTAT=OFF \ -DFLB_IN_WINEVTLOG=OFF \ -DFLB_IN_COLLECTD=OFF \ -DFLB_IN_PROMETHEUS_SCRAPE=OFF \ -DFLB_IN_STATSD=OFF \ -DFLB_IN_EVENT_TEST=OFF \ -DFLB_IN_STORAGE_BACKLOG=OFF \ -DFLB_IN_EMITTER=OFF \ -DFLB_IN_NODE_EXPORTER_METRICS=OFF \ -DFLB_IN_WINDOWS_EXPORTER_METRICS=OFF \ -DFLB_IN_OPENTELEMETRY=OFF \ -DFLB_OUT_AZURE=OFF \ -DFLB_OUT_AZURE_BLOB=OFF \ -DFLB_OUT_AZURE_KUSTO=OFF \ -DFLB_OUT_BIGQUERY=OFF \ -DFLB_OUT_CALYPTIA=OFF \ -DFLB_OUT_COUNTER=OFF \ -DFLB_OUT_DATADOG=OFF \ -DFLB_OUT_ES=OFF \ -DFLB_OUT_EXIT=OFF \ -DFLB_OUT_GELF=OFF \ -DFLB_OUT_HTTP=OFF \ -DFLB_OUT_INFLUXDB=OFF \ -DFLB_OUT_NATS=OFF \ -DFLB_OUT_NRLOGS=OFF \ -DFLB_OUT_OPENSEARCH=OFF \ -DFLB_OUT_TCP=OFF \ -DFLB_OUT_PLOT=OFF \ -DFLB_OUT_FILE=OFF \ -DFLB_OUT_TD=OFF \ -DFLB_OUT_RETRY=OFF \ -DFLB_OUT_PGSQL=OFF \ -DFLB_OUT_SKYWALKING=OFF \ -DFLB_OUT_SLACK=OFF \ -DFLB_OUT_SPLUNK=OFF \ -DFLB_OUT_STACKDRIVER=OFF \ -DFLB_OUT_STDOUT=OFF \ -DFLB_OUT_SYSLOG=OFF \ -DFLB_OUT_LIB=OFF \ -DFLB_OUT_NULL=OFF \ -DFLB_OUT_FLOWCOUNTER=OFF \ -DFLB_OUT_LOGDNA=OFF \ -DFLB_OUT_LOKI=OFF \ -DFLB_OUT_KAFKA=OFF \ -DFLB_OUT_KAFKA_REST=OFF \ -DFLB_OUT_CLOUDWATCH_LOGS=OFF \ -DFLB_OUT_KINESIS_FIREHOSE=OFF \ -DFLB_OUT_KINESIS_STREAMS=OFF \ -DFLB_OUT_OPENTELEMETRY=OFF \ -DFLB_OUT_PROMETHEUS_EXPORTER=OFF \ -DFLB_OUT_PROMETHEUS_REMOTE_WRITE=OFF \ -DFLB_OUT_S3=OFF \ -DFLB_OUT_WEBSOCKET=OFF \ -DFLB_FILTER_ALTER_SIZE=OFF \ -DFLB_FILTER_AWS=OFF \ -DFLB_FILTER_ECS=OFF \ -DFLB_FILTER_CHECKLIST=OFF \ -DFLB_FILTER_EXPECT=OFF \ -DFLB_FILTER_GREP=OFF \ -DFLB_FILTER_STDOUT=OFF \ -DFLB_FILTER_PARSER=OFF \ -DFLB_FILTER_KUBERNETES=OFF \ -DFLB_FILTER_REWRITE_TAG=OFF \ -DFLB_FILTER_THROTTLE=OFF \ -DFLB_FILTER_THROTTLE_SIZE=OFF \ -DFLB_FILTER_TYPE_CONVERTER=OFF \ -DFLB_FILTER_MULTILINE=OFF \ -DFLB_FILTER_NEST=OFF \ -DFLB_FILTER_LUA=OFF \ -DFLB_FILTER_LUA_USE_MPACK=OFF \ -DFLB_FILTER_TENSORFLOW=OFF \ -DFLB_FILTER_GEOIP2=OFF \ -DFLB_FILTER_NIGHTFALL=OFF \ -DFLB_FILTER_WASM=OFF \ ../

make -j8 -k

In file included from /home/user1/fluent-bit/tools/xxd-c/xxd-c.c:30: /home/user1/fluent-bit/lib/monkey/include/monkey/mk_core/mk_getopt.h:23:10: fatal error: mk_core/mk_core_info.h: No such file or directory 23 | #include <mk_core/mk_core_info.h>

In file included from /home/user1/fluent-bit/include/fluent-bit/flb_macros.h:23, from /home/user1/fluent-bit/include/fluent-bit/flb_sds.h:29, from /home/user1/fluent-bit/include/fluent-bit/flb_slist.h:24, from /home/user1/fluent-bit/src/record_accessor/flb_ra_parser.c:21: /home/user1/fluent-bit/lib/monkey/include/monkey/mk_core.h:33:10: fatal error: mk_core/mk_core_info.h: No such file or directory 33 | #include <mk_core/mk_core_info.h> | ^~~~~~~~ compilation terminated.

In file included from /home/user1/fluent-bit/include/fluent-bit/flb_macros.h:23, from /home/user1/fluent-bit/include/fluent-bit/flb_str.h:24, from ra.l:6: /home/user1/fluent-bit/lib/monkey/include/monkey/mk_core.h:33:10: fatal error: mk_core/mk_core_info.h: No such file or directory 33 | #include <mk_core/mk_core_info.h> | ^~~~~~~~ compilation terminated.

In file included from /home/user1/fluent-bit/include/fluent-bit/flb_macros.h:23, from /home/user1/fluent-bit/include/fluent-bit/flb_mem.h:28, from ra.y:14: /home/user1/fluent-bit/lib/monkey/include/monkey/mk_core.h:33:10: fatal error: mk_core/mk_core_info.h: No such file or directory 33 | #include <mk_core/mk_core_info.h> | ^~~~~~~~ compilation terminated.

In file included from /home/user1/fluent-bit/lib/monkey/include/monkey/mk_core/mk_utils.h:28, from /home/user1/fluent-bit/lib/monkey/mk_core/mk_rconf.c:31: /home/user1/fluent-bit/lib/monkey/include/monkey/mk_core/mk_pthread.h:23:10: fatal error: mk_core/mk_core_info.h: No such file or directory 23 | #include <mk_core/mk_core_info.h> | ^~~~~~~~ compilation terminated.

In file included from /home/user1/fluent-bit/lib/monkey/include/monkey/mk_core/mk_utils.h:28, from /home/user1/fluent-bit/lib/monkey/mk_core/mk_utils.c:31: /home/user1/fluent-bit/lib/monkey/include/monkey/mk_core/mk_pthread.h:23:10: fatal error: mk_core/mk_core_info.h: No such file or directory 23 | #include <mk_core/mk_core_info.h> | ^~~~~~~~ compilation terminated.



**Expected behavior**
It should compile, or reject invalid option selection

**Your Environment**
Building with v2.0.6 git tag

**Additional context**
nickbroon commented 1 year ago

Adding -DFLB_HTTP_SERVER=ON \ to the above cmake options, allows the build to get further, so I'm guessing the http_server isn't actually optional or dependent option of other options. Though it now fails with:

/home/nick/work/graphiant/graphnos/fluent-bit/src/flb_input_chunk.c:1736:38: warning: 'struct input_chunk_raw' declared inside parameter list will not be visible outside of this definition or declaration
 1736 | static void destroy_chunk_raw(struct input_chunk_raw *cr)
      |                                      ^~~~~~~~~~~~~~~
/home/nick/work/graphiant/graphnos/fluent-bit/src/flb_input_chunk.c: In function 'destroy_chunk_raw':
/home/nick/work/graphiant/graphnos/fluent-bit/src/flb_input_chunk.c:1738:11: error: invalid use of undefined type 'struct input_chunk_raw'
 1738 |     if (cr->buf_data) {
      |           ^~
/home/nick/work/graphiant/graphnos/fluent-bit/src/flb_input_chunk.c:1739:20: error: invalid use of undefined type 'struct input_chunk_raw'
 1739 |         flb_free(cr->buf_data);
      |                    ^~
/home/nick/work/graphiant/graphnos/fluent-bit/src/flb_input_chunk.c:1742:11: error: invalid use of undefined type 'struct input_chunk_raw'
 1742 |     if (cr->tag) {
      |           ^~
/home/nick/work/graphiant/graphnos/fluent-bit/src/flb_input_chunk.c:1743:27: error: invalid use of undefined type 'struct input_chunk_raw'
 1743 |         flb_sds_destroy(cr->tag);
      |                           ^~
/home/nick/work/graphiant/graphnos/fluent-bit/src/flb_input_chunk.c: In function 'append_to_ring_buffer':
/home/nick/work/graphiant/graphnos/fluent-bit/src/flb_input_chunk.c:1763:31: error: invalid application of 'sizeof' to incomplete type 'struct input_chunk_raw'
 1763 |     cr = flb_calloc(1, sizeof(struct input_chunk_raw));
      |                               ^~~~~~
/home/nick/work/graphiant/graphnos/fluent-bit/src/flb_input_chunk.c:1768:7: error: invalid use of undefined type 'struct input_chunk_raw'
 1768 |     cr->ins = ins;
      |       ^~

So it looks like the -DFLB_CHUNK_TRACE=ON \ is needed as well, and either is not really optional or is a dependency os some other options.

nickbroon commented 1 year ago

Actually it looks like DFLB_IN_STORAGE_BACKLOG=ON is also needed to fix this issue.

nickbroon commented 1 year ago

In the end I had to add -DFLB_CHUNK_TRACE=ON and -DFLB_HTTP_SERVER=ON to the original set of options to get fluent-bit to compile. So I suspect these are the options that contain bugs that prevent them being truly optional.

leonardo-albertovich commented 1 year ago

The problem with FLB_HTTP_SERVER is that it's used to build monkey which is a hard dependency from which a lot of the primitives come (ie. the event loop abstraction layer) but it's also used to toggle FLB_HAVE_HTTP_SERVER which is the actual switch to determine if the http server (used to expose metrics) is enabled.

So in that case I think a PR to decouple those would be great.

For FLB_IN_STORAGE_BACKLOG you probably realized that the structure input_chunk_raw was mistakenly defined inside the conditional that only executes if FLB_HAVE_IN_STORAGE_BACKLOG is defined so we need to move it up a little bit and that fixes the issue.

As for FLB_CHUNK_TRACE, I tried to build without it after fixing the previous issue (and cleaning the cmake build directory) and it worked as expected.

nickbroon commented 1 year ago

Yes, in the end I just had to FLB_IN_STORAGE_BACKLOG and FLB_HTTP_SERVER to cmake options to get my build to work. FLB_CHUNK_TRACE was just me getting confused as I tried to discover the root cause.

I've open an PR for the FLB_IN_STORAGE_BACKLOG issues. If I get some time I'll look a separating FLB_HTTP_SERVER and FLB_HAVE_HTTP_SERVER as it would be nice to be able to disable the monitoring http metric server.

github-actions[bot] commented 1 year ago

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days. Maintainers can add the exempt-stale label.

github-actions[bot] commented 1 year ago

This issue was closed because it has been stalled for 5 days with no activity.

mavam commented 1 year ago

Hitting the same bug:

/opt/homebrew/include/monkey/mk_core.h:33:10: fatal error: 'mk_core/mk_core_info.h' file not found
#include <mk_core/mk_core_info.h>

I'm on macOS and installed via brew install fluent-bit. The core issue is that the mk_core directory lacks the file mk_core_info.h:

mk_dep_unistd.h      mk_getopt.h          mk_rconf.h
mk_dirent.h          mk_iov.h             mk_sleep.h
mk_event.h           mk_limits.h          mk_string.h
mk_event_epoll.h     mk_list.h            mk_thread.h
mk_event_kqueue.h    mk_macros.h          mk_thread_channel.h
mk_event_libevent.h  mk_memory.h          mk_uio.h
mk_event_select.h    mk_pipe.h            mk_unistd.h
mk_file.h            mk_pthread.h         mk_utils.h

Since this is a generated file from a *.in file, it's easy to forget. Is there an issue tracking this already? The fix is probably a single line in CMake, by adding install(...) at the right location.

nickbroon commented 1 month ago

In light of https://github.com/fluent/fluent-bit/commit/9311b43a258352797af40749ab31a63c32acfd04 it would still be good to be able to build fluent-bit with the http-server disabled.

As @leonardo-albertovich mentioned in previous comment in this issue it appears that monkey is basically a full mandatory dependency, so i tried applying this patch which means its always included:

--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -610,11 +610,7 @@ MK_SET_OPTION(MK_SYSTEM_MALLOC  ON)
 MK_SET_OPTION(MK_DEBUG          ON)

 # Build Monkey HTTP Server
-if(FLB_HTTP_SERVER)
-  add_subdirectory(${FLB_PATH_LIB_MONKEY} EXCLUDE_FROM_ALL)
-else()
-  add_subdirectory(${FLB_PATH_LIB_MONKEY}/mk_core EXCLUDE_FROM_ALL)
-endif()
+add_subdirectory(${FLB_PATH_LIB_MONKEY} EXCLUDE_FROM_ALL)

and building with this set of build options, that turns of a lot plugins and importantly also turns off FLB_HTTP_SERVER:

    -DFLB_REGEX=ON \
    -DFLB_UTF8_ENCODER=ON \
    -DFLB_PARSER=ON \
    -DFLB_TLS=ON \
    -DFLB_BINARY=ON \
    -DFLB_SHARED_LIB=ON \
    -DFLB_INOTIFY=ON \
    -DFLB_BACKTRACE=ON \
    -DFLB_RECORD_ACCESSOR=ON \
    -DFLB_CONFIG_YAML=ON \
    -DFLB_SQLDB=ON \
    -DFLB_IN_SYSTEMD=ON \
    -DFLB_OUT_FORWARD=ON \
    -DFLB_FILTER_MODIFY=ON \
    -DFLB_FILTER_RECORD_MODIFIER=ON \
    -DFLB_HTTP_SERVER=OFF \
    -DFLB_IN_STORAGE_BACKLOG=ON \
    -DFLB_ALL=OFF \
    -DFLB_DEBUG=OFF \
    -DFLB_RELEASE=OFF \
    -DFLB_SMALL=OFF \
    -DFLB_COVERAGE=OFF \
    -DFLB_JEMALLOC=OFF \
    -DFLB_EXAMPLES=OFF \
    -DFLB_VALGRIND=OFF \
    -DFLB_TRACE=OFF \
    -DFLB_CHUNK_TRACE=OFF \
    -DFLB_TESTS_RUNTIME=OFF \
    -DFLB_TESTS_INTERNAL=OFF \
    -DFLB_TESTS_INTERNAL_FUZZ=OFF \
    -DFLB_TESTS_OSSFUZZ=OFF \
    -DFLB_MTRACE=OFF \
    -DFLB_POSIX_TLS=OFF \
    -DFLB_LUAJIT=OFF \
    -DFLB_SIGNV4=OFF \
    -DFLB_AWS=OFF \
    -DFLB_STATIC_CONF=OFF \
    -DFLB_STREAM_PROCESSOR=OFF \
    -DFLB_CORO_STACK_SIZE=OFF \
    -DFLB_AVRO_ENCODER=OFF \
    -DFLB_AWS_ERROR_REPORTER=OFF \
    -DFLB_ARROW=OFF \
    -DFLB_WINDOWS_DEFAULTS=OFF \
    -DFLB_WASM=OFF \
    -DFLB_WAMRC=OFF \
    -DFLB_METRICS=OFF \
    -DFLB_PROXY_GO=OFF \
    -DFLB_CUSTOM_CALYPTIA=OFF \
    -DFLB_IN_CPU=OFF \
    -DFLB_IN_THERMAL=OFF \
    -DFLB_IN_DISK=OFF \
    -DFLB_IN_DOCKER=OFF \
    -DFLB_IN_DOCKER_EVENTS=OFF \
    -DFLB_IN_ELASTICSEARCH=OFF \
    -DFLB_IN_EXEC=OFF \
    -DFLB_IN_EXEC_WASI=OFF \
    -DFLB_IN_EVENT_TEST=OFF \
    -DFLB_IN_EVENT_TYPE=OFF \
    -DFLB_IN_FLUENTBIT_METRICS=OFF \
    -DFLB_IN_FORWARD=OFF \
    -DFLB_IN_HEALTH=OFF \
    -DFLB_IN_HTTP=OFF \
    -DFLB_IN_MEM=OFF \
    -DFLB_IN_KAFKA=OFF \
    -DFLB_IN_KMSG=OFF \
    -DFLB_IN_LIB=OFF \
    -DFLB_IN_RANDOM=OFF \
    -DFLB_IN_SERIAL=OFF \
    -DFLB_IN_SPLUNK=OFF \
    -DFLB_IN_STDIN=OFF \
    -DFLB_IN_SYSLOG=OFF \
    -DFLB_IN_TAIL=OFF \
    -DFLB_IN_UDP=OFF \
    -DFLB_IN_TCP=OFF \
    -DFLB_IN_UNIX_SOCKET=OFF \
    -DFLB_IN_MQTT=OFF \
    -DFLB_IN_HEAD=OFF \
    -DFLB_IN_PROC=OFF \
    -DFLB_IN_DUMMY=OFF \
    -DFLB_IN_NGINX_EXPORTER_METRICS=OFF \
    -DFLB_IN_NETIF=OFF \
    -DFLB_IN_WINLOG=OFF \
    -DFLB_IN_WINSTAT=OFF \
    -DFLB_IN_WINEVTLOG=OFF \
    -DFLB_IN_COLLECTD=OFF \
    -DFLB_IN_PROMETHEUS_REMOTE_WRITE=OFF \
    -DFLB_IN_PROMETHEUS_SCRAPE=OFF \
    -DFLB_IN_STATSD=OFF \
    -DFLB_IN_EVENT_TEST=OFF \
    -DFLB_IN_EMITTER=OFF \
    -DFLB_IN_NODE_EXPORTER_METRICS=OFF \
    -DFLB_IN_WINDOWS_EXPORTER_METRICS=OFF \
    -DFLB_IN_OPENTELEMETRY=OFF \
    -DFLB_OUT_AZURE=OFF \
    -DFLB_OUT_AZURE_BLOB=OFF \
    -DFLB_OUT_AZURE_KUSTO=OFF \
    -DFLB_OUT_BIGQUERY=OFF \
    -DFLB_OUT_CALYPTIA=OFF \
    -DFLB_OUT_COUNTER=OFF \
    -DFLB_OUT_DATADOG=OFF \
    -DFLB_OUT_ES=OFF \
    -DFLB_OUT_EXIT=OFF \
    -DFLB_OUT_GELF=OFF \
    -DFLB_OUT_HTTP=OFF \
    -DFLB_OUT_INFLUXDB=OFF \
    -DFLB_OUT_NATS=OFF \
    -DFLB_OUT_NRLOGS=OFF \
    -DFLB_OUT_OPENSEARCH=OFF \
    -DFLB_OUT_TCP=OFF \
    -DFLB_OUT_PLOT=OFF \
    -DFLB_OUT_FILE=OFF \
    -DFLB_OUT_TD=OFF \
    -DFLB_OUT_RETRY=OFF \
    -DFLB_OUT_PGSQL=OFF \
    -DFLB_OUT_SKYWALKING=OFF \
    -DFLB_OUT_SLACK=OFF \
    -DFLB_OUT_SPLUNK=OFF \
    -DFLB_OUT_STACKDRIVER=OFF \
    -DFLB_OUT_STDOUT=OFF \
    -DFLB_OUT_SYSLOG=OFF \
    -DFLB_OUT_LIB=OFF \
    -DFLB_OUT_NULL=OFF \
    -DFLB_OUT_FLOWCOUNTER=OFF \
    -DFLB_OUT_LOGDNA=OFF \
    -DFLB_OUT_LOKI=OFF \
    -DFLB_OUT_KAFKA=OFF \
    -DFLB_OUT_KAFKA_REST=OFF \
    -DFLB_OUT_CLOUDWATCH_LOGS=OFF \
    -DFLB_OUT_KINESIS_FIREHOSE=OFF \
    -DFLB_OUT_KINESIS_STREAMS=OFF \
    -DFLB_OUT_OPENTELEMETRY=OFF \
    -DFLB_OUT_PROMETHEUS_EXPORTER=OFF \
    -DFLB_OUT_PROMETHEUS_REMOTE_WRITE=OFF \
    -DFLB_OUT_S3=OFF \
    -DFLB_OUT_VIVO_EXPORTER=OFF \
    -DFLB_OUT_WEBSOCKET=OFF \
    -DFLB_FILTER_ALTER_SIZE=OFF \
    -DFLB_FILTER_AWS=OFF \
    -DFLB_FILTER_ECS=OFF \
    -DFLB_FILTER_CHECKLIST=OFF \
    -DFLB_FILTER_EXPECT=OFF \
    -DFLB_FILTER_GREP=OFF \
    -DFLB_FILTER_STDOUT=OFF \
    -DFLB_FILTER_PARSER=OFF \
    -DFLB_FILTER_KUBERNETES=OFF \
    -DFLB_FILTER_REWRITE_TAG=OFF \
    -DFLB_FILTER_THROTTLE=OFF \
    -DFLB_FILTER_THROTTLE_SIZE=OFF \
    -DFLB_FILTER_TYPE_CONVERTER=OFF \
    -DFLB_FILTER_MULTILINE=OFF \
    -DFLB_FILTER_NEST=OFF \
    -DFLB_FILTER_LUA=OFF \
    -DFLB_FILTER_LUA_USE_MPACK=OFF \
    -DFLB_FILTER_TENSORFLOW=OFF \
    -DFLB_FILTER_GEOIP2=OFF \
    -DFLB_FILTER_NIGHTFALL=OFF \
    -DFLB_FILTER_WASM=OFF \

but the build still fails 😢 

/home/user1/fluent-bit-3.0.4/src/flb_input.c: In function ‘flb_input_instance_init’:
/home/user1/fluent-bit-3.0.4/src/flb_input.c:923:24: warning: unused variable ‘ctx’ [-Wunused-variable]
  923 |     struct flb_config *ctx = ins->config;
      |                        ^~~
/home/user1/fluent-bit-3.0.4/src/flb_input.c: In function ‘flb_input_ingestion_paused’:
/home/user1/fluent-bit-3.0.4/src/flb_input.c:1687:38: warning: initialization discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers]
 1687 |                       1, (char *[]) {flb_input_name(ins)});
      |                                      ^~~~~~~~~~~~~~
/home/user1/fluent-bit-3.0.4/src/flb_input.c: In function ‘flb_input_ingestion_resumed’:
/home/user1/fluent-bit-3.0.4/src/flb_input.c:1696:38: warning: initialization discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers]
 1696 |                       1, (char *[]) {flb_input_name(ins)});
      |                                      ^~~~~~~~~~~~~~
/home/user1/fluent-bit-3.0.4/src/flb_input_chunk.c: In function ‘flb_input_chunk_drop_task_route’:
/home/user1/fluent-bit-3.0.4/src/flb_input_chunk.c:364:55: error: ‘struct flb_task’ has no member named ‘records’
  364 |                 *dropped_record_count = (ssize_t) task->records;
      |                                                       ^~
/home/user1/fluent-bit-3.0.4/src/flb_input_chunk.c: In function ‘flb_input_chunk_map’:
/home/user1/fluent-bit-3.0.4/src/flb_input_chunk.c:599:14: warning: unused variable ‘ts’ [-Wunused-variable]
  599 |     uint64_t ts;
      |              ^~
/home/user1/fluent-bit-3.0.4/src/flb_input_chunk.c: In function ‘input_chunk_append_raw’:
/home/user1/fluent-bit-3.0.4/src/flb_input_chunk.c:1574:29: error: ‘struct flb_input_chunk’ has no member named ‘total_records’
 1574 |     total_records_start = ic->total_records;
      |                             ^~
/home/user1/fluent-bit-3.0.4/src/flb_input_chunk.c:1575:7: error: ‘struct flb_input_chunk’ has no member named ‘added_records’
 1575 |     ic->added_records =  n_records;
      |       ^~
/home/user1/fluent-bit-3.0.4/src/flb_input_chunk.c:1576:7: error: ‘struct flb_input_chunk’ has no member named ‘total_records’
 1576 |     ic->total_records += n_records;
      |       ^~
/home/user1/fluent-bit-3.0.4/src/flb_input_chunk.c:1618:11: error: ‘struct flb_input_chunk’ has no member named ‘added_records’
 1618 |         ic->added_records = 0;
      |           ^~
/home/user1/fluent-bit-3.0.4/src/flb_input_chunk.c:1619:11: error: ‘struct flb_input_chunk’ has no member named ‘total_records’
 1619 |         ic->total_records = total_records_start;
      |           ^~
/home/user1/fluent-bit-3.0.4/src/flb_input_chunk.c: At top level:
/home/user1/fluent-bit-3.0.4/src/flb_input_chunk.c:100:16: warning: ‘get_input_chunk_record_count’ defined but not used [-Wunused-function]
  100 | static ssize_t get_input_chunk_record_count(struct flb_input_chunk *input_chunk)
      |                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
make[3]: *** [src/CMakeFiles/fluent-bit-static.dir/build.make:359: src/CMakeFiles/fluent-bit-static.dir/flb_input_chunk.c.o] Error 1
make[3]: *** Waiting for unfinished jobs....
/home/user1/fluent-bit-3.0.4/src/flb_filter.c: In function ‘flb_filter_do’:
/home/user1/fluent-bit-3.0.4/src/flb_filter.c:129:5: error: ‘in_records’ undeclared (first use in this function)
  129 |     in_records = ic->added_records;
      |     ^~~~~~~~~~
/home/user1/fluent-bit-3.0.4/src/flb_filter.c:129:5: note: each undeclared identifier is reported only once for each function it appears in
/home/user1/fluent-bit-3.0.4/src/flb_filter.c:129:20: error: ‘struct flb_input_chunk’ has no member named ‘added_records’
  129 |     in_records = ic->added_records;
      |                    ^~
/home/user1/fluent-bit-3.0.4/src/flb_filter.c:130:5: error: ‘pre_records’ undeclared (first use in this function)
  130 |     pre_records = ic->total_records - in_records;
      |     ^~~~~~~~~~~
/home/user1/fluent-bit-3.0.4/src/flb_filter.c:130:21: error: ‘struct flb_input_chunk’ has no member named ‘total_records’
  130 |     pre_records = ic->total_records - in_records;
      |                     ^~
/home/user1/fluent-bit-3.0.4/src/flb_filter.c:204:23: error: ‘struct flb_input_chunk’ has no member named ‘total_records’
  204 |                     ic->total_records = pre_records;
      |                       ^~
/home/user1/fluent-bit-3.0.4/src/flb_filter.c:218:21: error: ‘out_records’ undeclared (first use in this function)
  218 |                     out_records = flb_mp_count(out_buf, out_size);
      |                     ^~~~~~~~~~~
/home/user1/fluent-bit-3.0.4/src/flb_filter.c:247:23: error: ‘struct flb_input_chunk’ has no member named ‘total_records’
  247 |                     ic->total_records = pre_records + in_records;
      |                       ^~
make[3]: *** [src/CMakeFiles/fluent-bit-static.dir/build.make:429: src/CMakeFiles/fluent-bit-static.dir/flb_filter.c.o] Error 1
make[3]: Leaving directory '/home/user1/fluent-bit-3.0.4/obj-x86_64-linux-gnu'
make[2]: *** [CMakeFiles/Makefile2:3583: src/CMakeFiles/fluent-bit-static.dir/all] Error 2
make[2]: Leaving directory '/home/user1/fluent-bit-3.0.4/obj-x86_64-linux-gnu'
make[1]: *** [Makefile:159: all] Error 2
make[1]: Leaving directory '/home/user1/fluent-bit-3.0.4/obj-x86_64-linux-gnu'

What is necessary to be able to disable the http-server at build time?

tguenneguez commented 1 month ago

I have the same problem in 3.0.6 version ...