manticoresoftware / manticoresearch

Easy to use open source fast database for search | Good alternative to Elasticsearch now | Drop-in replacement for E in the ELK soon
https://manticoresearch.com
GNU General Public License v3.0
9.07k stars 509 forks source link

Kafka integration fixes #2761

Closed PavelShilin89 closed 1 week ago

PavelShilin89 commented 1 week ago

Type of Change:

Description of the Change:

Related Issue (provide the link):

github-actions[bot] commented 1 week ago

clt-amd64

👎 CLT tests in test/clt-tests/integrations/test-integrations- ✅ OK: 3 ❌ Failed: 1 ⏳ Duration: 142s 👉 Check Action Results for commit 2d2a89d Failed tests:

test/clt-tests/integrations/test-integrations-fluentbit.rec ```diff ––– input ––– set -b ––– output ––– ––– input ––– export PATH=/usr/bin:/usr/local/bin:/usr/sbin:/sbin:/bin ––– output ––– ––– input ––– apt-get update > /dev/null 2>&1; echo $? ––– output ––– 0 ––– input ––– apt-get -y install gnupg2 openjdk-8-jre-headless curl jq > /dev/null 2>&1; echo $? ––– output ––– 0 ––– input ––– mkdir -p /usr/share/fluentbit/fluentbit && cd /usr/share/fluentbit ––– output ––– ––– input ––– echo -e "[SERVICE]\n flush 1\n daemon On\n log_level info\n\n[INPUT]\n name tail\n path /var/log/dpkg.log\n inotify_watcher false\n read_from_head true\n\n[OUTPUT]\n name es\n match *\n host 127.0.0.1\n port 9308\n index dpkg_log" > /usr/share/fluentbit/fluentbit/fluentbit.conf ––– output ––– ––– input ––– bash -c "$(curl -s https://raw.githubusercontent.com/fluent/fluent-bit/master/install.sh)" > /dev/null 2>&1;echo $? ––– output ––– 0 ––– input ––– ln -s /opt/fluent-bit/bin/fluent-bit /usr/bin/fluent-bit ––– output ––– ––– input ––– fluent-bit --version | grep 'Fluent Bit' ––– output ––– - Fluent Bit v#!/[0-9]{1}\.[0-9]{1}\.[0-9]{2}/!# + Fluent Bit v3.2.0 ––– input ––– echo -e "2023-05-31 10:42:55 status triggers-awaited ca-certificates-java:all 20190405ubuntu1.1\n2023-05-31 10:42:55 trigproc libc-bin:amd64 2.31-0ubuntu9.9 \n2023-05-31 10:42:55 status half-configured libc-bin:amd64 2.31-0ubuntu9.9\n2023-05-31 10:42:55 status installed libc-bin:amd64 2.31-0ubuntu9.9\n2023-05-31 10:42:55 trigproc systemd:amd64 245.4-4ubuntu3.21 " > /var/log/dpkg.log; echo $? ––– output ––– 0 ––– input ––– cat /var/log/dpkg.log | wc -l ––– output ––– 5 ––– input ––– rm -f /var/log/manticore/searchd.log; searchd --stopwait > /dev/null; searchd; if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore/searchd.log;fi ––– output ––– Manticore %{SEMVER} %{COMMITDATE}#!/(\sdev)?\s/!#(columnar %{SEMVER} %{COMMITDATE}) (secondary %{SEMVER} %{COMMITDATE}) (knn %{SEMVER} %{COMMITDATE}) Copyright (c) 2001-2016, Andrew Aksyonoff Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com) Copyright (c) 2017-%{YEAR}, Manticore Software LTD (https://manticoresearch.com) [#!/[0-9]{2}:[0-9]{2}.[0-9]{3}/!#] [#!/[0-9]+/!#] using config file '/etc/manticoresearch/manticore.conf' (%{NUMBER} chars)... starting daemon version '%{SEMVER} %{COMMITDATE}#!/(\sdev)?\s/!#(columnar %{SEMVER} %{COMMITDATE}) (secondary %{SEMVER} %{COMMITDATE}) (knn %{SEMVER} %{COMMITDATE})' ... listening on %{IPADDR}:9312 for sphinx and http(s) listening on %{IPADDR}:9306 for mysql listening on %{IPADDR}:9308 for sphinx and http(s) Buddy started! ––– input ––– fluent-bit -c /usr/share/fluentbit/fluentbit/fluentbit.conf > /dev/null 2>&1; echo $? ––– output ––– 0 ––– input ––– sleep 3; curl -s localhost:9308/cli_json -d 'describe dpkg_log' | jq -cM '.[].data | sort_by(.Field)' ––– output ––– [{"Field":"@timestamp","Type":"timestamp","Properties":""},{"Field":"id","Type":"bigint","Properties":""},{"Field":"log","Type":"text","Properties":"indexed stored"}] ––– input ––– mysql -P9306 -h0 -N -e "SHOW TABLES\G; SELECT COUNT(*) FROM dpkg_log\G; SELECT log FROM dpkg_log\G;" | grep -v '^*' ––– output ––– dpkg_log rt 5 2023-05-31 10:42:55 status triggers-awaited ca-certificates-java:all 20190405ubuntu1.1 2023-05-31 10:42:55 trigproc libc-bin:amd64 2.31-0ubuntu9.9 2023-05-31 10:42:55 status half-configured libc-bin:amd64 2.31-0ubuntu9.9 2023-05-31 10:42:55 status installed libc-bin:amd64 2.31-0ubuntu9.9 2023-05-31 10:42:55 trigproc systemd:amd64 245.4-4ubuntu3.21 ```