crowdsecurity / crowdsec

CrowdSec - the open-source and participative security solution offering crowdsourced protection against malicious IPs and access to the most advanced real-world CTI.
https://crowdsec.net
MIT License
8.3k stars 429 forks source link

cscli explain : exit status 2 #2231

Closed g00g1 closed 1 year ago

g00g1 commented 1 year ago

What happened?

cscli crashed while running explain mode

What did you expect to happen?

cscli do not crash

How can we reproduce it (as minimally and precisely as possible)?

Details

```console admin@flopster /var/log $ sudo ip netns exec crowdsec-test cscli explain -f /var/log/mail.log --type syslog WARN[25-05-2023 14:01:19] log file contains 54885 lines. This may take lot of resources. time="25-05-2023 14:01:19" level=info msg="single file mode : log_media=stdout daemonize=false" time="25-05-2023 14:01:19" level=info msg="Enabled feature flags: " time="25-05-2023 14:01:19" level=info msg="Crowdsec v1.5.1-linux-eddb994c0b48d77b34a3f22b719dc5716670d2ae" time="25-05-2023 14:01:19" level=warning msg="MaxOpenConns is 0, defaulting to 100" time="25-05-2023 14:01:19" level=info msg="Loading prometheus collectors" time="25-05-2023 14:01:19" level=warning msg="Exprhelpers loaded without database client." time="25-05-2023 14:01:19" level=info msg="Loading grok library /etc/crowdsec/patterns" time="25-05-2023 14:01:20" level=info msg="Loading enrich plugins" time="25-05-2023 14:01:20" level=info msg="Successfully registered enricher 'GeoIpCity'" time="25-05-2023 14:01:20" level=info msg="Successfully registered enricher 'GeoIpASN'" time="25-05-2023 14:01:20" level=info msg="Successfully registered enricher 'IpToRange'" time="25-05-2023 14:01:20" level=info msg="Successfully registered enricher 'reverse_dns'" time="25-05-2023 14:01:20" level=info msg="Successfully registered enricher 'ParseDate'" time="25-05-2023 14:01:20" level=info msg="Successfully registered enricher 'UnmarshalJSON'" time="25-05-2023 14:01:20" level=info msg="Loading parsers from 14 files" time="25-05-2023 14:01:20" level=info msg="Loaded 2 parser nodes" file=/etc/crowdsec/parsers/s00-raw/syslog-logs.yaml stage=s00-raw time="25-05-2023 14:01:20" level=info msg="Loaded 1 parser nodes" file=/etc/crowdsec/parsers/s01-parse/dovecot-logs.yaml stage=s01-parse time="25-05-2023 14:01:20" level=info msg="Loaded 1 parser nodes" file=/etc/crowdsec/parsers/s01-parse/endlessh-logs.yaml stage=s01-parse time="25-05-2023 14:01:20" level=info msg="Loaded 1 parser nodes" file=/etc/crowdsec/parsers/s01-parse/nginx-logs.yaml stage=s01-parse time="25-05-2023 14:01:20" level=info msg="Loaded 1 parser nodes" file=/etc/crowdsec/parsers/s01-parse/postfix-logs.yaml stage=s01-parse time="25-05-2023 14:01:20" level=info msg="Loaded 1 parser nodes" file=/etc/crowdsec/parsers/s01-parse/postscreen-logs.yaml stage=s01-parse time="25-05-2023 14:01:20" level=info msg="Loaded 1 parser nodes" file=/etc/crowdsec/parsers/s01-parse/sshd-logs.yaml stage=s01-parse time="25-05-2023 14:01:20" level=info msg="Loaded 1 parser nodes" file=/etc/crowdsec/parsers/s01-parse/zz-g00g1-dovecot.yaml stage=s01-parse time="25-05-2023 14:01:20" level=info msg="Loaded 1 parser nodes" file=/etc/crowdsec/parsers/s01-parse/zz-g00g1-nginx.yaml stage=s01-parse time="25-05-2023 14:01:20" level=info msg="Loaded 3 parser nodes" file=/etc/crowdsec/parsers/s01-parse/zz-g00g1-postfix.yaml stage=s01-parse time="25-05-2023 14:01:20" level=info msg="Loaded 1 parser nodes" file=/etc/crowdsec/parsers/s02-enrich/dateparse-enrich.yaml stage=s02-enrich time="25-05-2023 14:01:20" level=info msg="Loaded 1 parser nodes" file=/etc/crowdsec/parsers/s02-enrich/geoip-enrich.yaml stage=s02-enrich time="25-05-2023 14:01:20" level=info msg="Loaded 1 parser nodes" file=/etc/crowdsec/parsers/s02-enrich/http-logs.yaml stage=s02-enrich time="25-05-2023 14:01:20" level=info msg="Loaded 1 parser nodes" file=/etc/crowdsec/parsers/s02-enrich/whitelists.yaml stage=s02-enrich time="25-05-2023 14:01:20" level=info msg="Loaded 17 nodes from 3 stages" time="25-05-2023 14:01:20" level=info msg="No postoverflow parsers to load" time="25-05-2023 14:01:20" level=info msg="Loading 42 scenario files" time="25-05-2023 14:01:20" level=info msg="Adding trigger bucket" cfg=cool-pond file=/etc/crowdsec/scenarios/vmware-vcenter-vmsa-2021-0027.yaml name=crowdsecurity/vmware-vcenter-vmsa-2021-0027 time="25-05-2023 14:01:20" level=info msg="Adding trigger bucket" cfg=wild-feather file=/etc/crowdsec/scenarios/CVE-2022-37042.yaml name=crowdsecurity/CVE-2022-37042 time="25-05-2023 14:01:20" level=info msg="Adding leaky bucket" cfg=hidden-morning file=/etc/crowdsec/scenarios/http-probing.yaml name=crowdsecurity/http-probing time="25-05-2023 14:01:20" level=info msg="Adding trigger bucket" cfg=purple-mountain file=/etc/crowdsec/scenarios/jira_cve-2021-26086.yaml name=crowdsecurity/jira_cve-2021-26086 time="25-05-2023 14:01:20" level=info msg="Adding leaky bucket" cfg=crimson-snow file=/etc/crowdsec/scenarios/ssh-bf.yaml name=crowdsecurity/ssh-bf time="25-05-2023 14:01:20" level=info msg="Adding leaky bucket" cfg=blue-pond file=/etc/crowdsec/scenarios/ssh-bf.yaml name=crowdsecurity/ssh-bf_user-enum time="25-05-2023 14:01:20" level=info msg="Adding trigger bucket" cfg=little-feather file=/etc/crowdsec/scenarios/pulse-secure-sslvpn-cve-2019-11510.yaml name=crowdsecurity/pulse-secure-sslvpn-cve-2019-11510 time="25-05-2023 14:01:20" level=info msg="Adding leaky bucket" cfg=holy-voice file=/etc/crowdsec/scenarios/ssh-slow-bf.yaml name=crowdsecurity/ssh-slow-bf time="25-05-2023 14:01:20" level=info msg="Adding leaky bucket" cfg=red-voice file=/etc/crowdsec/scenarios/ssh-slow-bf.yaml name=crowdsecurity/ssh-slow-bf_user-enum time="25-05-2023 14:01:20" level=info msg="Adding trigger bucket" cfg=billowing-voice file=/etc/crowdsec/scenarios/spring4shell_cve-2022-22965.yaml name=crowdsecurity/spring4shell_cve-2022-22965 time="25-05-2023 14:01:20" level=info msg="Adding leaky bucket" cfg=solitary-firefly file=/etc/crowdsec/scenarios/http-backdoors-attempts.yaml name=crowdsecurity/http-backdoors-attempts time="25-05-2023 14:01:20" level=info msg="Adding trigger bucket" cfg=muddy-leaf file=/etc/crowdsec/scenarios/zz-g00g1-endlessh.yaml name=g00g1/endlessh-trap time="25-05-2023 14:01:20" level=info msg="Adding trigger bucket" cfg=shy-field file=/etc/crowdsec/scenarios/http-w00tw00t.yaml name=ltsich/http-w00tw00t time="25-05-2023 14:01:20" level=info msg="Adding leaky bucket" cfg=black-smoke file=/etc/crowdsec/scenarios/dovecot-spam.yaml name=crowdsecurity/dovecot-spam time="25-05-2023 14:01:20" level=info msg="Adding trigger bucket" cfg=polished-grass file=/etc/crowdsec/scenarios/thinkphp-cve-2018-20062.yaml name=crowdsecurity/thinkphp-cve-2018-20062 time="25-05-2023 14:01:20" level=info msg="Adding trigger bucket" cfg=hidden-field file=/etc/crowdsec/scenarios/zz-g00g1-dovecot.yaml name=g00g1/dovecot-bf time="25-05-2023 14:01:20" level=info msg="Adding trigger bucket" cfg=small-voice file=/etc/crowdsec/scenarios/grafana-cve-2021-43798.yaml name=crowdsecurity/grafana-cve-2021-43798 time="25-05-2023 14:01:20" level=info msg="Adding trigger bucket" cfg=aged-wildflower file=/etc/crowdsec/scenarios/CVE-2022-41082.yaml name=crowdsecurity/CVE-2022-41082 time="25-05-2023 14:01:20" level=info msg="Adding trigger bucket" cfg=throbbing-hill file=/etc/crowdsec/scenarios/CVE-2022-44877.yaml name=crowdsecurity/CVE-2022-44877 time="25-05-2023 14:01:20" level=info msg="Adding trigger bucket" cfg=young-silence file=/etc/crowdsec/scenarios/zz-g00g1-nginx.yaml name=g00g1/nginx-honeypot time="25-05-2023 14:01:20" level=info msg="Adding leaky bucket" cfg=bitter-hill file=/etc/crowdsec/scenarios/postfix-spam.yaml name=crowdsecurity/postfix-spam time="25-05-2023 14:01:20" level=info msg="Adding trigger bucket" cfg=dawn-wood file=/etc/crowdsec/scenarios/postfix-spam.yaml name=crowdsecurity/postscreen-rbl time="25-05-2023 14:01:20" level=info msg="Adding leaky bucket" cfg=restless-morning file=/etc/crowdsec/scenarios/http-sensitive-files.yaml name=crowdsecurity/http-sensitive-files time="25-05-2023 14:01:20" level=info msg="Adding trigger bucket" cfg=morning-fire file=/etc/crowdsec/scenarios/CVE-2022-26134.yaml name=crowdsecurity/CVE-2022-26134 time="25-05-2023 14:01:20" level=info msg="Adding leaky bucket" cfg=spring-night file=/etc/crowdsec/scenarios/endlessh-bf.yaml name=crowdsecurity/endlessh-bf time="25-05-2023 14:01:20" level=info msg="Adding leaky bucket" cfg=proud-river file=/etc/crowdsec/scenarios/http-xss-probing.yaml name=crowdsecurity/http-xss-probbing time="25-05-2023 14:01:20" level=info msg="Adding trigger bucket" cfg=blue-mountain file=/etc/crowdsec/scenarios/http-cve-2021-41773.yaml name=crowdsecurity/http-cve-2021-41773 time="25-05-2023 14:01:20" level=info msg="Adding trigger bucket" cfg=sparkling-moon file=/etc/crowdsec/scenarios/fortinet-cve-2018-13379.yaml name=crowdsecurity/fortinet-cve-2018-13379 time="25-05-2023 14:01:20" level=info msg="Adding trigger bucket" cfg=bold-forest file=/etc/crowdsec/scenarios/apache_log4j2_cve-2021-44228.yaml name=crowdsecurity/apache_log4j2_cve-2021-44228 time="25-05-2023 14:01:20" level=info msg="Adding leaky bucket" cfg=billowing-sound file=/etc/crowdsec/scenarios/http-generic-bf.yaml name=crowdsecurity/http-generic-bf time="25-05-2023 14:01:20" level=info msg="Adding leaky bucket" cfg=morning-glade file=/etc/crowdsec/scenarios/http-generic-bf.yaml name=LePresidente/http-generic-401-bf time="25-05-2023 14:01:20" level=info msg="Adding leaky bucket" cfg=fragrant-sound file=/etc/crowdsec/scenarios/http-generic-bf.yaml name=LePresidente/http-generic-403-bf time="25-05-2023 14:01:20" level=info msg="Adding leaky bucket" cfg=red-field file=/etc/crowdsec/scenarios/CVE-2022-46169.yaml name=crowdsecurity/CVE-2022-46169-bf time="25-05-2023 14:01:20" level=info msg="Adding trigger bucket" cfg=bold-butterfly file=/etc/crowdsec/scenarios/CVE-2022-46169.yaml name=crowdsecurity/CVE-2022-46169-cmd time="25-05-2023 14:01:20" level=info msg="Adding leaky bucket" cfg=cool-meadow file=/etc/crowdsec/scenarios/nginx-req-limit-exceeded.yaml name=crowdsecurity/nginx-req-limit-exceeded time="25-05-2023 14:01:20" level=info msg="Adding trigger bucket" cfg=cool-thunder file=/etc/crowdsec/scenarios/zz-g00g1-postfix.yaml name=g00g1/postfix-bf time="25-05-2023 14:01:20" level=info msg="Adding trigger bucket" cfg=old-frost file=/etc/crowdsec/scenarios/zz-g00g1-postfix.yaml name=g00g1/postfix-spam time="25-05-2023 14:01:20" level=info msg="Adding trigger bucket" cfg=solitary-glade file=/etc/crowdsec/scenarios/CVE-2022-40684.yaml name=crowdsecurity/fortinet-cve-2022-40684 time="25-05-2023 14:01:20" level=info msg="Adding trigger bucket" cfg=twilight-meadow file=/etc/crowdsec/scenarios/http-open-proxy.yaml name=crowdsecurity/http-open-proxy time="25-05-2023 14:01:20" level=info msg="Adding leaky bucket" cfg=long-feather file=/etc/crowdsec/scenarios/http-crawl-non_statics.yaml name=crowdsecurity/http-crawl-non_statics time="25-05-2023 14:01:20" level=info msg="Adding leaky bucket" cfg=ancient-voice file=/etc/crowdsec/scenarios/http-sqli-probing.yaml name=crowdsecurity/http-sqli-probbing-detection time="25-05-2023 14:01:20" level=info msg="Adding trigger bucket" cfg=spring-sky file=/etc/crowdsec/scenarios/f5-big-ip-cve-2020-5902.yaml name=crowdsecurity/f5-big-ip-cve-2020-5902 time="25-05-2023 14:01:20" level=info msg="Adding leaky bucket" cfg=bitter-waterfall file=/etc/crowdsec/scenarios/http-path-traversal-probing.yaml name=crowdsecurity/http-path-traversal-probing time="25-05-2023 14:01:20" level=info msg="Adding trigger bucket" cfg=red-paper file=/etc/crowdsec/scenarios/vmware-cve-2022-22954.yaml name=crowdsecurity/vmware-cve-2022-22954 time="25-05-2023 14:01:20" level=info msg="Adding trigger bucket" cfg=damp-paper file=/etc/crowdsec/scenarios/CVE-2019-18935.yaml name=crowdsecurity/CVE-2019-18935 time="25-05-2023 14:01:20" level=info msg="Adding trigger bucket" cfg=polished-dawn file=/etc/crowdsec/scenarios/CVE-2022-42889.yaml name=crowdsecurity/CVE-2022-42889 time="25-05-2023 14:01:20" level=info msg="Adding trigger bucket" cfg=red-night file=/etc/crowdsec/scenarios/CVE-2022-35914.yaml name=crowdsecurity/CVE-2022-35914 time="25-05-2023 14:01:20" level=info msg="Adding leaky bucket" cfg=ancient-meadow file=/etc/crowdsec/scenarios/CVE-2022-41697.yaml name=crowdsecurity/CVE-2022-41697 time="25-05-2023 14:01:20" level=info msg="Adding trigger bucket" cfg=proud-glade file=/etc/crowdsec/scenarios/http-cve-2021-42013.yaml name=crowdsecurity/http-cve-2021-42013 time="25-05-2023 14:01:20" level=warning msg="Loaded 49 scenarios" time="25-05-2023 14:01:20" level=info msg="Adding file /var/log/mail.log to filelist" type="file:///var/log/mail.log" time="25-05-2023 14:01:20" level=info msg="Starting processing data" time="25-05-2023 14:01:20" level=info msg="reading /var/log/mail.log at once" type="file:///var/log/mail.log" time="25-05-2023 14:01:20" level=error msg="error while performing request: dial tcp 127.0.0.1:8080: connect: connection refused; 4 retries left" time="25-05-2023 14:01:20" level=info msg="retrying in 16 seconds (attempt 2 of 5)" fatal error: concurrent map read and map write goroutine 35 [running]: github.com/crowdsecurity/crowdsec/pkg/parser.Parse({{_, _}, {_, _, _}, _, {_, _}}, {0x0, 0x1, ...}, ...) github.com/crowdsecurity/crowdsec/pkg/parser/runtime.go:329 +0x1128 main.runParse(0xc000453920, 0xc0004538c0?, {{0xc001bea960, 0x0}, {0xc000dc41c0, 0x3, 0x4}, 0x1, {0xc0009f1920, 0x16}}, ...) github.com/crowdsecurity/crowdsec/cmd/crowdsec/parse.go:33 +0x3a5 main.runCrowdsec.func1.1() github.com/crowdsecurity/crowdsec/cmd/crowdsec/crowdsec.go:57 +0xe8 gopkg.in/tomb%2ev2.(*Tomb).run(0x30c82a0, 0xc002240050?) gopkg.in/tomb.v2@v2.0.0-20161208151619-d5d1b5820637/tomb.go:163 +0x36 created by gopkg.in/tomb%2ev2.(*Tomb).Go gopkg.in/tomb.v2@v2.0.0-20161208151619-d5d1b5820637/tomb.go:159 +0xee goroutine 1 [chan receive]: main.Serve(0xc00221e3f0, 0x1?, 0x1e483e1?) github.com/crowdsecurity/crowdsec/cmd/crowdsec/serve.go:370 +0x7c5 main.StartRunSvc() github.com/crowdsecurity/crowdsec/cmd/crowdsec/run_in_svc.go:61 +0x3b5 main.main() github.com/crowdsecurity/crowdsec/cmd/crowdsec/main.go:336 +0x225 goroutine 50 [select]: database/sql.(*DB).connectionOpener(0xc0021a1e10, {0x225e030, 0xc002240000}) database/sql/sql.go:1218 +0x8d created by database/sql.OpenDB database/sql/sql.go:791 +0x18d goroutine 37 [runnable]: github.com/crowdsecurity/crowdsec/pkg/parser.(*Node).process(0xc00198eab8, 0xc001bda000, {{0xc001bea960, 0x0}, {0xc000dc41c0, 0x3, 0x4}, 0x1, {0xc0009f1920, 0x16}}, ...) github.com/crowdsecurity/crowdsec/pkg/parser/node.go:284 +0xeab github.com/crowdsecurity/crowdsec/pkg/parser.(*Node).process(0xc00198f320, 0xc001bda000, {{0xc001bea960, 0x0}, {0xc000dc41c0, 0x3, 0x4}, 0x1, {0xc0009f1920, 0x16}}, ...) github.com/crowdsecurity/crowdsec/pkg/parser/node.go:361 +0x1f85 github.com/crowdsecurity/crowdsec/pkg/parser.Parse({{_, _}, {_, _, _}, _, {_, _}}, {0x0, 0x1, ...}, ...) github.com/crowdsecurity/crowdsec/pkg/parser/runtime.go:322 +0x102d main.runParse(0xc000453920, 0xc0004538c0?, {{0xc001bea960, 0x0}, {0xc000dc41c0, 0x3, 0x4}, 0x1, {0xc0009f1920, 0x16}}, ...) github.com/crowdsecurity/crowdsec/cmd/crowdsec/parse.go:33 +0x3a5 main.runCrowdsec.func1.1() github.com/crowdsecurity/crowdsec/cmd/crowdsec/crowdsec.go:57 +0xe8 gopkg.in/tomb%2ev2.(*Tomb).run(0x30c82a0, 0xc002240050?) gopkg.in/tomb.v2@v2.0.0-20161208151619-d5d1b5820637/tomb.go:163 +0x36 created by gopkg.in/tomb%2ev2.(*Tomb).Go gopkg.in/tomb.v2@v2.0.0-20161208151619-d5d1b5820637/tomb.go:159 +0xee goroutine 36 [runnable]: regexp.(*inputString).step(0xc000042390?, 0x6?) regexp/regexp.go:391 +0x8d regexp.(*machine).match(0xc0000422d0, {0x2260e80, 0xc000042390}, 0x0) regexp/exec.go:237 +0x49d regexp.(*Regexp).doExecute(0xc000d55900, {0x0?, 0x0}, {0x0, 0x0, 0x0}, {0xc001454d48, 0xba}, 0x4?, 0x13e, ...) regexp/exec.go:542 +0x319 regexp.(*Regexp).FindStringSubmatch(0xc000d55900, {0xc001454d48, 0xba}) regexp/regexp.go:1046 +0x8f github.com/crowdsecurity/grokky.(*PatternLegacy).Parse(0xc002285760, {0xc001454d48?, 0xc001f0fd50?}) github.com/crowdsecurity/grokky@v0.2.1/pattern_legacy.go:14 +0x45 github.com/crowdsecurity/crowdsec/pkg/parser.(*Node).process(0xc001f10ab8, 0xc00105a000, {{0xc001bea960, 0x0}, {0xc000dc41c0, 0x3, 0x4}, 0x1, {0xc0009f1920, 0x16}}, ...) github.com/crowdsecurity/crowdsec/pkg/parser/node.go:288 +0xf2d github.com/crowdsecurity/crowdsec/pkg/parser.(*Node).process(0xc001f11320, 0xc00105a000, {{0xc001bea960, 0x0}, {0xc000dc41c0, 0x3, 0x4}, 0x1, {0xc0009f1920, 0x16}}, ...) github.com/crowdsecurity/crowdsec/pkg/parser/node.go:361 +0x1f85 github.com/crowdsecurity/crowdsec/pkg/parser.Parse({{_, _}, {_, _, _}, _, {_, _}}, {0x0, 0x1, ...}, ...) github.com/crowdsecurity/crowdsec/pkg/parser/runtime.go:322 +0x102d main.runParse(0xc000453920, 0xc0004538c0?, {{0xc001bea960, 0x0}, {0xc000dc41c0, 0x3, 0x4}, 0x1, {0xc0009f1920, 0x16}}, ...) github.com/crowdsecurity/crowdsec/cmd/crowdsec/parse.go:33 +0x3a5 main.runCrowdsec.func1.1() github.com/crowdsecurity/crowdsec/cmd/crowdsec/crowdsec.go:57 +0xe8 gopkg.in/tomb%2ev2.(*Tomb).run(0x30c82a0, 0xc002240050?) gopkg.in/tomb.v2@v2.0.0-20161208151619-d5d1b5820637/tomb.go:163 +0x36 created by gopkg.in/tomb%2ev2.(*Tomb).Go gopkg.in/tomb.v2@v2.0.0-20161208151619-d5d1b5820637/tomb.go:159 +0xee goroutine 135 [chan send]: github.com/crowdsecurity/crowdsec/pkg/acquisition/modules/file.(*FileSource).readFile(0xc00197c360, {0x7ffd52e0368e, 0x11}, 0x0?, 0x30c80c0) github.com/crowdsecurity/crowdsec/pkg/acquisition/modules/file/file.go:538 +0x8e8 github.com/crowdsecurity/crowdsec/pkg/acquisition/modules/file.(*FileSource).OneShotAcquisition(0xc00197c360, 0x24?, 0xc002152bd0?) github.com/crowdsecurity/crowdsec/pkg/acquisition/modules/file/file.go:267 +0x210 github.com/crowdsecurity/crowdsec/pkg/acquisition.StartAcquisition.func1() github.com/crowdsecurity/crowdsec/pkg/acquisition/acquisition.go:320 +0x43b gopkg.in/tomb%2ev2.(*Tomb).run(0x30c80c0, 0xc00196ec00?) gopkg.in/tomb.v2@v2.0.0-20161208151619-d5d1b5820637/tomb.go:163 +0x36 created by gopkg.in/tomb%2ev2.(*Tomb).Go gopkg.in/tomb.v2@v2.0.0-20161208151619-d5d1b5820637/tomb.go:159 +0xee goroutine 38 [select]: main.runPour(0xc0004538c0, {0xc0017a8000, 0x31, 0x49}, 0xc002e73050?, 0xc00221e3f0) github.com/crowdsecurity/crowdsec/cmd/crowdsec/pour.go:20 +0xfd main.runCrowdsec.func2.1() github.com/crowdsecurity/crowdsec/cmd/crowdsec/crowdsec.go:83 +0x6c gopkg.in/tomb%2ev2.(*Tomb).run(0x30c8180, 0xc002240050?) gopkg.in/tomb.v2@v2.0.0-20161208151619-d5d1b5820637/tomb.go:163 +0x36 created by gopkg.in/tomb%2ev2.(*Tomb).Go gopkg.in/tomb.v2@v2.0.0-20161208151619-d5d1b5820637/tomb.go:159 +0xee goroutine 130 [chan receive]: gopkg.in/tomb%2ev2.(*Tomb).Wait(0x30c80c0) gopkg.in/tomb.v2@v2.0.0-20161208151619-d5d1b5820637/tomb.go:126 +0x35 github.com/crowdsecurity/crowdsec/pkg/acquisition.StartAcquisition({0xc00196d420, 0x1, 0xc000082f30?}, 0xc000453920, 0x30c80c0) github.com/crowdsecurity/crowdsec/pkg/acquisition/acquisition.go:332 +0x1ed main.runCrowdsec(0xc00221e3f0, 0xc002d54300) github.com/crowdsecurity/crowdsec/cmd/crowdsec/crowdsec.go:125 +0x32d main.serveCrowdsec.func1.1() github.com/crowdsecurity/crowdsec/cmd/crowdsec/crowdsec.go:141 +0xe5 created by main.serveCrowdsec.func1 github.com/crowdsecurity/crowdsec/cmd/crowdsec/crowdsec.go:136 +0xd8 goroutine 16 [select]: main.waitOnTomb() github.com/crowdsecurity/crowdsec/cmd/crowdsec/crowdsec.go:227 +0x7d main.serveCrowdsec.func1() github.com/crowdsecurity/crowdsec/cmd/crowdsec/crowdsec.go:150 +0xdd gopkg.in/tomb%2ev2.(*Tomb).run(0x30c81e0, 0xc002240050?) gopkg.in/tomb.v2@v2.0.0-20161208151619-d5d1b5820637/tomb.go:163 +0x36 created by gopkg.in/tomb%2ev2.(*Tomb).Go gopkg.in/tomb.v2@v2.0.0-20161208151619-d5d1b5820637/tomb.go:159 +0xee goroutine 114 [IO wait]: internal/poll.runtime_pollWait(0x7f5f2024ed00, 0x72) runtime/netpoll.go:306 +0x89 internal/poll.(*pollDesc).wait(0xc001696000?, 0xc000083c80?, 0x0) internal/poll/fd_poll_runtime.go:84 +0x32 internal/poll.(*pollDesc).waitRead(...) internal/poll/fd_poll_runtime.go:89 internal/poll.(*FD).Accept(0xc001696000) internal/poll/fd_unix.go:614 +0x2bd net.(*netFD).accept(0xc001696000) net/fd_unix.go:172 +0x35 net.(*TCPListener).accept(0xc0021ae468) net/tcpsock_posix.go:148 +0x25 net.(*TCPListener).Accept(0xc0021ae468) net/tcpsock.go:297 +0x3d net/http.(*Server).Serve(0xc0016460f0, {0x225c700, 0xc0021ae468}) net/http/server.go:3059 +0x385 net/http.(*Server).ListenAndServe(0xc0016460f0) net/http/server.go:2988 +0x7d net/http.ListenAndServe(...) net/http/server.go:3242 main.servePrometheus(0xc001b9eff0, 0xc00227b3e0?, 0x2260a40?, 0xc00227b3e0?) github.com/crowdsecurity/crowdsec/cmd/crowdsec/metrics.go:196 +0x1fc created by main.StartRunSvc github.com/crowdsecurity/crowdsec/cmd/crowdsec/run_in_svc.go:59 +0x398 goroutine 146 [select]: github.com/crowdsecurity/crowdsec/pkg/apiclient.retryRoundTripper.RoundTrip({{0x224d140, 0x2faa240}, 0x5, {0xc000376030, 0x3, 0x3}, 0x1, 0x0}, 0xc00201a300) github.com/crowdsecurity/crowdsec/pkg/apiclient/auth.go:108 +0x250 github.com/crowdsecurity/crowdsec/pkg/apiclient.retryRoundTripper.RoundTrip({{0x2249e40, 0xc0020040c0}, 0x2, {0xc000618140, 0x2, 0x2}, 0x0, 0xc002000160}, 0xc00201a200) github.com/crowdsecurity/crowdsec/pkg/apiclient/auth.go:118 +0x2ae github.com/crowdsecurity/crowdsec/pkg/apiclient.(*JWTTransport).RoundTrip(0xc0010800a0, 0xc00201a200) github.com/crowdsecurity/crowdsec/pkg/apiclient/auth.go:258 +0x54d net/http.send(0xc00201a200, {0x2249e20, 0xc0010800a0}, {0x8?, 0x1de4bc0?, 0x0?}) net/http/client.go:252 +0x5f7 net/http.(*Client).send(0xc00131e000, 0xc00201a200, {0x4053d4?, 0x9?, 0x0?}) net/http/client.go:176 +0x9b net/http.(*Client).do(0xc00131e000, 0xc00201a200) net/http/client.go:716 +0x8fb net/http.(*Client).Do(...) net/http/client.go:582 github.com/crowdsecurity/crowdsec/pkg/apiclient.(*ApiClient).Do(0xc00200e080, {0x225e068, 0xc0000460b0}, 0xc00201a100, {0x1c4a0a0, 0xc00131e060}) github.com/crowdsecurity/crowdsec/pkg/apiclient/client_http.go:65 +0x325 github.com/crowdsecurity/crowdsec/pkg/apiclient.(*AuthService).AuthenticateWatcher(0xc00200e088, {0x225e068, 0xc0000460b0}, {0xc001a84020, 0xc002000010, {0xc000075400, 0x2a, 0x40}}) github.com/crowdsecurity/crowdsec/pkg/apiclient/auth_service.go:63 +0x1a5 main.runOutput(0xc0004538c0?, 0xc000535860, 0xc002e73050, {{0xc0004cc8d0, 0x0}, {0x0, 0x0, 0x0}, 0x1, {0xc0009f1920, ...}}, ...) github.com/crowdsecurity/crowdsec/cmd/crowdsec/output.go:100 +0x68e main.runCrowdsec.func3.1() github.com/crowdsecurity/crowdsec/cmd/crowdsec/crowdsec.go:101 +0x166 gopkg.in/tomb%2ev2.(*Tomb).run(0x30c8240, 0xc00227b3e0?) gopkg.in/tomb.v2@v2.0.0-20161208151619-d5d1b5820637/tomb.go:163 +0x36 created by gopkg.in/tomb%2ev2.(*Tomb).Go gopkg.in/tomb.v2@v2.0.0-20161208151619-d5d1b5820637/tomb.go:159 +0xee FATA[25-05-2023 14:01:20] fail to run crowdsec for test: exit status 2 ```

Anything else we need to know?

No response

Crowdsec version

```console admin@flopster /var/log $ cscli version 2023/05/25 14:03:23 version: v1.5.1-eddb994c0b48d77b34a3f22b719dc5716670d2ae 2023/05/25 14:03:23 Codename: alphaga 2023/05/25 14:03:23 BuildDate: 2023-05-17_09:37:10 2023/05/25 14:03:23 GoVersion: 1.20.4 2023/05/25 14:03:23 Platform: linux 2023/05/25 14:03:23 Constraint_parser: >= 1.0, <= 2.0 2023/05/25 14:03:23 Constraint_scenario: >= 1.0, < 3.0 2023/05/25 14:03:23 Constraint_api: v1 2023/05/25 14:03:23 Constraint_acquis: >= 1.0, < 2.0 ```

OS version

```console $ cat /etc/os-release NAME=Gentoo ID=gentoo PRETTY_NAME="Gentoo Linux" ANSI_COLOR="1;32" HOME_URL="https://www.gentoo.org/" SUPPORT_URL="https://www.gentoo.org/support/" BUG_REPORT_URL="https://bugs.gentoo.org/" VERSION_ID="2.13" $ uname -a Linux flopster 6.1.27 #1 SMP PREEMPT_DYNAMIC Wed May 17 12:26:07 EEST 2023 x86_64 AMD Ryzen 5 3600 6-Core Processor AuthenticAMD GNU/Linux ```

Enabled collections and parsers

```console $ cscli hub list -o raw crowdsecurity/base-http-scenarios,"enabled,tainted",0.6,http common : scanners detection,collections crowdsecurity/dovecot,"enabled,tainted",0.1,dovecot support : parser and spammer detection,collections crowdsecurity/endlessh,enabled,0.1,endlessh support : logs parser and brute-force detection,collections crowdsecurity/http-cve,enabled,2.0,,collections crowdsecurity/linux,enabled,0.2,core linux support : syslog+geoip+ssh,collections crowdsecurity/nginx,"enabled,tainted",0.2,nginx support : parser and generic http scenarios,collections crowdsecurity/postfix,"enabled,tainted",0.2,postfix support : parser and spammer detection,collections crowdsecurity/sshd,enabled,0.2,sshd support : parser and brute-force detection,collections nginx-custom.yaml,"enabled,local",n/a,,collections postfix-better.yaml,"enabled,local",n/a,,collections crowdsecurity/dateparse-enrich,enabled,0.2,,parsers crowdsecurity/dovecot-logs,"enabled,tainted",?,Parse dovecot logs,parsers crowdsecurity/endlessh-logs,enabled,0.2,Parse Endlessh logs,parsers crowdsecurity/geoip-enrich,enabled,0.2,"Populate event with geoloc info : as, country, coords, source range.",parsers crowdsecurity/http-logs,enabled,1.1,"Parse more Specifically HTTP logs, such as HTTP Code, HTTP path, HTTP args and if its a static ressource",parsers crowdsecurity/nginx-logs,enabled,1.3,Parse nginx access and error logs,parsers crowdsecurity/postfix-logs,"enabled,tainted",?,Parse postfix logs,parsers crowdsecurity/postscreen-logs,enabled,0.2,Parse postscreen logs,parsers crowdsecurity/sshd-logs,enabled,2.0,Parse openSSH logs,parsers crowdsecurity/syslog-logs,enabled,0.8,,parsers crowdsecurity/whitelists,enabled,0.2,Whitelist events from private ipv4 addresses,parsers zz-g00g1-dovecot.yaml,"enabled,local",n/a,,parsers zz-g00g1-nginx.yaml,"enabled,local",n/a,,parsers zz-g00g1-postfix.yaml,"enabled,local",n/a,,parsers crowdsecurity/CVE-2019-18935,enabled,0.1,Detect Telerik CVE-2019-18935 exploitation attempts,scenarios crowdsecurity/CVE-2022-26134,enabled,0.1,Detect CVE-2022-26134 exploits,scenarios crowdsecurity/CVE-2022-35914,enabled,0.1,Detect CVE-2022-35914 exploits,scenarios crowdsecurity/CVE-2022-37042,enabled,0.1,Detect CVE-2022-37042 exploits,scenarios crowdsecurity/CVE-2022-40684,enabled,0.2,Detect cve-2022-40684 exploitation attempts,scenarios crowdsecurity/CVE-2022-41082,enabled,0.3,Detect CVE-2022-41082 exploits,scenarios crowdsecurity/CVE-2022-41697,enabled,0.1,Detect CVE-2022-41697 enumeration,scenarios crowdsecurity/CVE-2022-42889,enabled,0.2,Detect CVE-2022-42889 exploits (Text4Shell),scenarios crowdsecurity/CVE-2022-44877,enabled,0.2,Detect CVE-2022-44877 exploits,scenarios crowdsecurity/CVE-2022-46169,enabled,0.1,Detect CVE-2022-46169 brute forcing,scenarios crowdsecurity/apache_log4j2_cve-2021-44228,enabled,0.4,Detect cve-2021-44228 exploitation attemps,scenarios crowdsecurity/dovecot-spam,enabled,0.3,detect errors on dovecot,scenarios crowdsecurity/endlessh-bf,enabled,0.1,Detect SSH bruteforce caught by Endlessh,scenarios crowdsecurity/f5-big-ip-cve-2020-5902,enabled,0.1,Detect cve-2020-5902 exploitation attemps,scenarios crowdsecurity/fortinet-cve-2018-13379,enabled,0.2,Detect cve-2018-13379 exploitation attemps,scenarios crowdsecurity/grafana-cve-2021-43798,enabled,0.1,Detect cve-2021-43798 exploitation attemps,scenarios crowdsecurity/http-backdoors-attempts,enabled,0.3,Detect attempt to common backdoors,scenarios crowdsecurity/http-crawl-non_statics,enabled,0.3,Detect aggressive crawl from single ip,scenarios crowdsecurity/http-cve-2021-41773,enabled,0.1,cve-2021-41773,scenarios crowdsecurity/http-cve-2021-42013,enabled,0.1,cve-2021-42013,scenarios crowdsecurity/http-generic-bf,enabled,0.4,Detect generic http brute force,scenarios crowdsecurity/http-open-proxy,enabled,0.3,Detect scan for open proxy,scenarios crowdsecurity/http-path-traversal-probing,enabled,0.2,Detect path traversal attempt,scenarios crowdsecurity/http-probing,enabled,0.2,Detect site scanning/probing from a single ip,scenarios crowdsecurity/http-sensitive-files,enabled,0.2,"Detect attempt to access to sensitive files (.log, .db ..) or folders (.git)",scenarios crowdsecurity/http-sqli-probing,enabled,0.2,A scenario that detects SQL injection probing with minimal false positives,scenarios crowdsecurity/http-xss-probing,enabled,0.2,A scenario that detects XSS probing with minimal false positives,scenarios crowdsecurity/jira_cve-2021-26086,enabled,0.1,Detect Atlassian Jira CVE-2021-26086 exploitation attemps,scenarios crowdsecurity/nginx-req-limit-exceeded,enabled,0.1,Detects IPs which violate nginx's user set request limit.,scenarios crowdsecurity/postfix-spam,"enabled,tainted",?,Detect spammers,scenarios crowdsecurity/pulse-secure-sslvpn-cve-2019-11510,enabled,0.2,Detect cve-2019-11510 exploitation attemps,scenarios crowdsecurity/spring4shell_cve-2022-22965,enabled,0.2,Detect cve-2022-22965 probing,scenarios crowdsecurity/ssh-bf,enabled,0.1,Detect ssh bruteforce,scenarios crowdsecurity/ssh-slow-bf,enabled,0.2,Detect slow ssh bruteforce,scenarios crowdsecurity/thinkphp-cve-2018-20062,enabled,0.3,Detect ThinkPHP CVE-2018-20062 exploitation attemps,scenarios crowdsecurity/vmware-cve-2022-22954,enabled,0.2,Detect Vmware CVE-2022-22954 exploitation attempts,scenarios crowdsecurity/vmware-vcenter-vmsa-2021-0027,enabled,0.1,Detect VMSA-2021-0027 exploitation attemps,scenarios ltsich/http-w00tw00t,enabled,0.1,detect w00tw00t,scenarios zz-g00g1-dovecot.yaml,"enabled,local",n/a,,scenarios zz-g00g1-endlessh.yaml,"enabled,local",n/a,,scenarios zz-g00g1-nginx.yaml,"enabled,local",n/a,,scenarios zz-g00g1-postfix.yaml,"enabled,local",n/a,,scenarios ```

Acquisition config

Details

```console $ cat /etc/crowdsec/acquis.yaml /etc/crowdsec/acquis.d/* filenames: - /var/log/auth.log labels: type: syslog --- filenames: - /var/log/mail.log labels: type: syslog --- filenames: - /var/log/endlessh.log labels: type: endlessh --- filenames: - /var/log/nginx-trap.log labels: type: nginx-custom --- filenames: - /var/log/nginx/*_log labels: type: nginx --- cat: '/etc/crowdsec/acquis.d/*': No such file or directory ```

Config show

```console $ sudo cscli config show Global: - Configuration Folder : /etc/crowdsec - Configuration Folder : /etc/crowdsec - Data Folder : /var/lib/crowdsec/data - Hub Folder : /etc/crowdsec/hub - Simulation File : /etc/crowdsec/simulation.yaml - Log Folder : /var/log/ - Log level : info - Log Media : file Crowdsec: - Acquisition File : /etc/crowdsec/acquis.yaml - Parsers routines : 3 - Acquisition Folder : /etc/crowdsec/acquis.d cscli: - Output : human - Hub Branch : - Hub Folder : /etc/crowdsec/hub API Client: - URL : http://127.0.0.1:8080/ - Login : bc9630b07f7c3d6fb32ffb5164071c81knwaPq7bxOLQLmLq - Credentials File : /etc/crowdsec/local_api_credentials.yaml Local API Server: - Listen URL : 127.0.0.1:8080 - Profile File : /etc/crowdsec/profiles.yaml - Trusted IPs: - 127.0.0.1 - ::1 - Database: - Type : sqlite - Path : /var/lib/crowdsec/data/crowdsec.db - Flush age : 7d - Flush size : 5000 ```

Prometheus metrics

```console $ sudo cscli metrics -o raw file:/var/log/auth.log: reads: 1609 unparsed: 1609 file:/var/log/endlessh.log: reads: 1 unparsed: 1 file:/var/log/mail.log: parsed: 2 pour: 2 reads: 2 file:/var/log/nginx/access_log: parsed: 10 pour: 4 reads: 10 child-crowdsecurity/endlessh-logs: hits: 2 unparsed: 2 child-crowdsecurity/http-logs: hits: 30 parsed: 20 unparsed: 10 child-crowdsecurity/nginx-logs: hits: 10 parsed: 10 child-crowdsecurity/postfix-logs: hits: 4 parsed: 2 unparsed: 2 child-crowdsecurity/syslog-logs: hits: 1611 parsed: 1611 crowdsecurity/dateparse-enrich: hits: 12 parsed: 12 crowdsecurity/endlessh-logs: hits: 1 unparsed: 1 crowdsecurity/geoip-enrich: hits: 12 parsed: 12 crowdsecurity/http-logs: hits: 10 parsed: 10 crowdsecurity/nginx-logs: hits: 10 parsed: 10 crowdsecurity/non-syslog: hits: 11 parsed: 11 crowdsecurity/postfix-logs: hits: 2 parsed: 2 crowdsecurity/syslog-logs: hits: 1611 parsed: 1611 crowdsecurity/whitelists: hits: 12 parsed: 12 crowdsecurity/http-crawl-non_statics: curr_count: 0 instantiation: 4 pour: 4 underflow: 4 crowdsecurity/postfix-spam: curr_count: 0 instantiation: 1 pour: 2 underflow: 1 /v1/alerts: GET: 522 /v1/decisions/stream: GET: 78 /v1/heartbeat: GET: 12 /v1/watchers/login: POST: 525 cs-firewall-bouncer-1684573093: /v1/decisions/stream: GET: 78 bc9630b07f7c3d6fb32ffb5164071c81knwaPq7bxOLQLmLq: /v1/alerts: GET: 522 /v1/heartbeat: GET: 12 {} crowdsecurity/CVE-2019-18935: CAPI: ban: 37 crowdsecurity/CVE-2022-26134: CAPI: ban: 213 crowdsecurity/CVE-2022-35914: CAPI: ban: 50 crowdsecurity/CVE-2022-37042: CAPI: ban: 17 crowdsecurity/CVE-2022-41082: CAPI: ban: 790 crowdsecurity/CVE-2022-42889: CAPI: ban: 9 crowdsecurity/apache_log4j2_cve-2021-44228: CAPI: ban: 430 crowdsecurity/dovecot-spam: CAPI: ban: 2278 crowdsecurity/endlessh-bf: CAPI: ban: 1327 crowdsecurity/f5-big-ip-cve-2020-5902: CAPI: ban: 26 crowdsecurity/fortinet-cve-2018-13379: CAPI: ban: 127 crowdsecurity/grafana-cve-2021-43798: CAPI: ban: 14 crowdsecurity/http-backdoors-attempts: CAPI: ban: 380 crowdsecurity/http-bad-user-agent: CAPI: ban: 1993 crowdsecurity/http-crawl-non_statics: CAPI: ban: 1063 crowdsecurity/http-cve-2021-41773: CAPI: ban: 17 crowdsecurity/http-cve-2021-42013: CAPI: ban: 2 crowdsecurity/http-generic-bf: CAPI: ban: 30 crowdsecurity/http-open-proxy: CAPI: ban: 648 crowdsecurity/http-path-traversal-probing: CAPI: ban: 184 crowdsecurity/http-probing: CAPI: ban: 4225 crowdsecurity/http-sensitive-files: CAPI: ban: 8 crowdsecurity/jira_cve-2021-26086: CAPI: ban: 6 crowdsecurity/mysql-bf: CAPI: ban: 13 crowdsecurity/nginx-req-limit-exceeded: CAPI: ban: 110 crowdsecurity/postfix-spam: CAPI: ban: 1228 crowdsecurity/spring4shell_cve-2022-22965: CAPI: ban: 1 crowdsecurity/ssh-bf: CAPI: ban: 11686 crowdsecurity/ssh-slow-bf: CAPI: ban: 8 crowdsecurity/thinkphp-cve-2018-20062: CAPI: ban: 12 crowdsecurity/vmware-vcenter-vmsa-2021-0027: CAPI: ban: 1 g00g1/endlessh-trap: crowdsec: ban: 2 g00g1/nginx-honeypot: crowdsec: ban: 12 ltsich/http-w00tw00t: CAPI: ban: 2 crowdsecurity/dovecot-spam: 31 crowdsecurity/endlessh-bf: 2 crowdsecurity/http-probing: 2 crowdsecurity/postfix-spam: 330 g00g1/endlessh-trap: 80 g00g1/nginx-honeypot: 330 g00g1/postfix-bf: 1455 g00g1/postfix-spam: 1 {} ```

Related custom configs versions (if applicable) : notification plugins, custom scenarios, parsers etc.

github-actions[bot] commented 1 year ago

@g00g1: Thanks for opening an issue, it is currently awaiting triage.

In the meantime, you can:

  1. Check Crowdsec Documentation to see if your issue can be self resolved.
  2. You can also join our Discord.
  3. Check Releases to make sure your agent is on the latest version.
Details I am a bot created to help the [crowdsecurity](https://github.com/crowdsecurity) developers manage community feedback and contributions. You can check out my [manifest file](https://github.com/crowdsecurity/crowdsec/blob/master/.github/governance.yml) to understand my behavior and what I can do. If you want to use this for your project, you can check out the [BirthdayResearch/oss-governance-bot](https://github.com/BirthdayResearch/oss-governance-bot) repository.
LaurenceJJones commented 1 year ago

It seems it crashed because CrowdSec the service isn't running where the explain was run.

g00g1 commented 1 year ago

It seems it crashed because CrowdSec the service isn't running where the explain was run.

It is running on that machine. When I run in inside default netns, it warns that binding to 127.0.0.1:6060 failed

LaurenceJJones commented 1 year ago

But it got a connection refused from LAPI?

time="25-05-2023 14:01:20" level=error msg="error while performing request: dial tcp 127.0.0.1:8080: connect: 
g00g1 commented 1 year ago

@LaurenceJJones, this is what I see when running without netns with running crowdsec daemon:

Details

```console admin@flopster /var/log $ sudo cscli explain --crowdsec /usr/local/bin/crowdsec -f /var/log/mail.log --type syslog Password: WARN[25-05-2023 14:17:43] log file contains 54923 lines. This may take lot of resources. time="25-05-2023 14:17:43" level=info msg="single file mode : log_media=stdout daemonize=false" time="25-05-2023 14:17:43" level=info msg="Enabled feature flags: " time="25-05-2023 14:17:43" level=info msg="Crowdsec v1.5.1-linux-eddb994c0b48d77b34a3f22b719dc5716670d2ae" time="25-05-2023 14:17:43" level=warning msg="MaxOpenConns is 0, defaulting to 100" time="25-05-2023 14:17:43" level=info msg="Loading prometheus collectors" time="25-05-2023 14:17:43" level=warning msg="Exprhelpers loaded without database client." time="25-05-2023 14:17:43" level=info msg="Loading grok library /etc/crowdsec/patterns" time="25-05-2023 14:17:44" level=info msg="Loading enrich plugins" time="25-05-2023 14:17:44" level=info msg="Successfully registered enricher 'GeoIpCity'" time="25-05-2023 14:17:44" level=info msg="Successfully registered enricher 'GeoIpASN'" time="25-05-2023 14:17:44" level=info msg="Successfully registered enricher 'IpToRange'" time="25-05-2023 14:17:44" level=info msg="Successfully registered enricher 'reverse_dns'" time="25-05-2023 14:17:44" level=info msg="Successfully registered enricher 'ParseDate'" time="25-05-2023 14:17:44" level=info msg="Successfully registered enricher 'UnmarshalJSON'" time="25-05-2023 14:17:44" level=info msg="Loading parsers from 14 files" time="25-05-2023 14:17:44" level=info msg="Loaded 2 parser nodes" file=/etc/crowdsec/parsers/s00-raw/syslog-logs.yaml stage=s00-raw time="25-05-2023 14:17:44" level=info msg="Loaded 1 parser nodes" file=/etc/crowdsec/parsers/s01-parse/dovecot-logs.yaml stage=s01-parse time="25-05-2023 14:17:44" level=info msg="Loaded 1 parser nodes" file=/etc/crowdsec/parsers/s01-parse/endlessh-logs.yaml stage=s01-parse time="25-05-2023 14:17:44" level=info msg="Loaded 1 parser nodes" file=/etc/crowdsec/parsers/s01-parse/nginx-logs.yaml stage=s01-parse time="25-05-2023 14:17:44" level=info msg="Loaded 1 parser nodes" file=/etc/crowdsec/parsers/s01-parse/postfix-logs.yaml stage=s01-parse time="25-05-2023 14:17:44" level=info msg="Loaded 1 parser nodes" file=/etc/crowdsec/parsers/s01-parse/postscreen-logs.yaml stage=s01-parse time="25-05-2023 14:17:44" level=info msg="Loaded 1 parser nodes" file=/etc/crowdsec/parsers/s01-parse/sshd-logs.yaml stage=s01-parse time="25-05-2023 14:17:44" level=info msg="Loaded 1 parser nodes" file=/etc/crowdsec/parsers/s01-parse/zz-g00g1-dovecot.yaml stage=s01-parse time="25-05-2023 14:17:44" level=info msg="Loaded 1 parser nodes" file=/etc/crowdsec/parsers/s01-parse/zz-g00g1-nginx.yaml stage=s01-parse time="25-05-2023 14:17:44" level=info msg="Loaded 3 parser nodes" file=/etc/crowdsec/parsers/s01-parse/zz-g00g1-postfix.yaml stage=s01-parse time="25-05-2023 14:17:44" level=info msg="Loaded 1 parser nodes" file=/etc/crowdsec/parsers/s02-enrich/dateparse-enrich.yaml stage=s02-enrich time="25-05-2023 14:17:44" level=info msg="Loaded 1 parser nodes" file=/etc/crowdsec/parsers/s02-enrich/geoip-enrich.yaml stage=s02-enrich time="25-05-2023 14:17:44" level=info msg="Loaded 1 parser nodes" file=/etc/crowdsec/parsers/s02-enrich/http-logs.yaml stage=s02-enrich time="25-05-2023 14:17:44" level=info msg="Loaded 1 parser nodes" file=/etc/crowdsec/parsers/s02-enrich/whitelists.yaml stage=s02-enrich time="25-05-2023 14:17:44" level=info msg="Loaded 17 nodes from 3 stages" time="25-05-2023 14:17:44" level=info msg="No postoverflow parsers to load" time="25-05-2023 14:17:44" level=info msg="Loading 42 scenario files" time="25-05-2023 14:17:44" level=info msg="Adding trigger bucket" cfg=damp-sea file=/etc/crowdsec/scenarios/apache_log4j2_cve-2021-44228.yaml name=crowdsecurity/apache_log4j2_cve-2021-44228 time="25-05-2023 14:17:44" level=info msg="Adding leaky bucket" cfg=bold-bird file=/etc/crowdsec/scenarios/http-sqli-probing.yaml name=crowdsecurity/http-sqli-probbing-detection time="25-05-2023 14:17:44" level=info msg="Adding leaky bucket" cfg=weathered-meadow file=/etc/crowdsec/scenarios/http-crawl-non_statics.yaml name=crowdsecurity/http-crawl-non_statics time="25-05-2023 14:17:44" level=info msg="Adding trigger bucket" cfg=small-glitter file=/etc/crowdsec/scenarios/CVE-2022-44877.yaml name=crowdsecurity/CVE-2022-44877 time="25-05-2023 14:17:44" level=info msg="Adding leaky bucket" cfg=silent-sea file=/etc/crowdsec/scenarios/ssh-slow-bf.yaml name=crowdsecurity/ssh-slow-bf time="25-05-2023 14:17:44" level=info msg="Adding leaky bucket" cfg=wispy-voice file=/etc/crowdsec/scenarios/ssh-slow-bf.yaml name=crowdsecurity/ssh-slow-bf_user-enum time="25-05-2023 14:17:44" level=info msg="Adding leaky bucket" cfg=small-sunset file=/etc/crowdsec/scenarios/http-probing.yaml name=crowdsecurity/http-probing time="25-05-2023 14:17:44" level=info msg="Adding leaky bucket" cfg=twilight-haze file=/etc/crowdsec/scenarios/http-sensitive-files.yaml name=crowdsecurity/http-sensitive-files time="25-05-2023 14:17:44" level=info msg="Adding trigger bucket" cfg=red-glitter file=/etc/crowdsec/scenarios/zz-g00g1-nginx.yaml name=g00g1/nginx-honeypot time="25-05-2023 14:17:44" level=info msg="Adding trigger bucket" cfg=green-silence file=/etc/crowdsec/scenarios/pulse-secure-sslvpn-cve-2019-11510.yaml name=crowdsecurity/pulse-secure-sslvpn-cve-2019-11510 time="25-05-2023 14:17:44" level=info msg="Adding trigger bucket" cfg=aged-paper file=/etc/crowdsec/scenarios/vmware-cve-2022-22954.yaml name=crowdsecurity/vmware-cve-2022-22954 time="25-05-2023 14:17:44" level=info msg="Adding leaky bucket" cfg=twilight-frog file=/etc/crowdsec/scenarios/CVE-2022-41697.yaml name=crowdsecurity/CVE-2022-41697 time="25-05-2023 14:17:44" level=info msg="Adding trigger bucket" cfg=dawn-cloud file=/etc/crowdsec/scenarios/thinkphp-cve-2018-20062.yaml name=crowdsecurity/thinkphp-cve-2018-20062 time="25-05-2023 14:17:44" level=info msg="Adding trigger bucket" cfg=wandering-silence file=/etc/crowdsec/scenarios/CVE-2022-42889.yaml name=crowdsecurity/CVE-2022-42889 time="25-05-2023 14:17:44" level=info msg="Adding leaky bucket" cfg=icy-butterfly file=/etc/crowdsec/scenarios/endlessh-bf.yaml name=crowdsecurity/endlessh-bf time="25-05-2023 14:17:44" level=info msg="Adding trigger bucket" cfg=dawn-voice file=/etc/crowdsec/scenarios/jira_cve-2021-26086.yaml name=crowdsecurity/jira_cve-2021-26086 time="25-05-2023 14:17:44" level=info msg="Adding trigger bucket" cfg=still-wildflower file=/etc/crowdsec/scenarios/CVE-2022-26134.yaml name=crowdsecurity/CVE-2022-26134 time="25-05-2023 14:17:44" level=info msg="Adding trigger bucket" cfg=throbbing-pond file=/etc/crowdsec/scenarios/CVE-2022-40684.yaml name=crowdsecurity/fortinet-cve-2022-40684 time="25-05-2023 14:17:44" level=info msg="Adding leaky bucket" cfg=winter-sunset file=/etc/crowdsec/scenarios/dovecot-spam.yaml name=crowdsecurity/dovecot-spam time="25-05-2023 14:17:44" level=info msg="Adding trigger bucket" cfg=weathered-field file=/etc/crowdsec/scenarios/f5-big-ip-cve-2020-5902.yaml name=crowdsecurity/f5-big-ip-cve-2020-5902 time="25-05-2023 14:17:44" level=info msg="Adding trigger bucket" cfg=dark-glade file=/etc/crowdsec/scenarios/vmware-vcenter-vmsa-2021-0027.yaml name=crowdsecurity/vmware-vcenter-vmsa-2021-0027 time="25-05-2023 14:17:44" level=info msg="Adding leaky bucket" cfg=silent-smoke file=/etc/crowdsec/scenarios/http-path-traversal-probing.yaml name=crowdsecurity/http-path-traversal-probing time="25-05-2023 14:17:44" level=info msg="Adding leaky bucket" cfg=floral-thunder file=/etc/crowdsec/scenarios/CVE-2022-46169.yaml name=crowdsecurity/CVE-2022-46169-bf time="25-05-2023 14:17:44" level=info msg="Adding trigger bucket" cfg=throbbing-shadow file=/etc/crowdsec/scenarios/CVE-2022-46169.yaml name=crowdsecurity/CVE-2022-46169-cmd time="25-05-2023 14:17:44" level=info msg="Adding trigger bucket" cfg=icy-wildflower file=/etc/crowdsec/scenarios/CVE-2022-37042.yaml name=crowdsecurity/CVE-2022-37042 time="25-05-2023 14:17:44" level=info msg="Adding trigger bucket" cfg=icy-frog file=/etc/crowdsec/scenarios/spring4shell_cve-2022-22965.yaml name=crowdsecurity/spring4shell_cve-2022-22965 time="25-05-2023 14:17:44" level=info msg="Adding leaky bucket" cfg=delicate-sun file=/etc/crowdsec/scenarios/ssh-bf.yaml name=crowdsecurity/ssh-bf time="25-05-2023 14:17:44" level=info msg="Adding leaky bucket" cfg=frosty-water file=/etc/crowdsec/scenarios/ssh-bf.yaml name=crowdsecurity/ssh-bf_user-enum time="25-05-2023 14:17:44" level=info msg="Adding trigger bucket" cfg=late-voice file=/etc/crowdsec/scenarios/http-w00tw00t.yaml name=ltsich/http-w00tw00t time="25-05-2023 14:17:44" level=info msg="Adding trigger bucket" cfg=icy-glitter file=/etc/crowdsec/scenarios/zz-g00g1-endlessh.yaml name=g00g1/endlessh-trap time="25-05-2023 14:17:44" level=info msg="Adding leaky bucket" cfg=ancient-glade file=/etc/crowdsec/scenarios/nginx-req-limit-exceeded.yaml name=crowdsecurity/nginx-req-limit-exceeded time="25-05-2023 14:17:44" level=info msg="Adding leaky bucket" cfg=summer-sun file=/etc/crowdsec/scenarios/http-backdoors-attempts.yaml name=crowdsecurity/http-backdoors-attempts time="25-05-2023 14:17:44" level=info msg="Adding leaky bucket" cfg=still-field file=/etc/crowdsec/scenarios/http-xss-probing.yaml name=crowdsecurity/http-xss-probbing time="25-05-2023 14:17:44" level=info msg="Adding trigger bucket" cfg=wispy-glade file=/etc/crowdsec/scenarios/CVE-2022-41082.yaml name=crowdsecurity/CVE-2022-41082 time="25-05-2023 14:17:44" level=info msg="Adding trigger bucket" cfg=empty-firefly file=/etc/crowdsec/scenarios/zz-g00g1-postfix.yaml name=g00g1/postfix-bf time="25-05-2023 14:17:44" level=info msg="Adding trigger bucket" cfg=crimson-hill file=/etc/crowdsec/scenarios/zz-g00g1-postfix.yaml name=g00g1/postfix-spam time="25-05-2023 14:17:44" level=info msg="Adding trigger bucket" cfg=dark-forest file=/etc/crowdsec/scenarios/CVE-2022-35914.yaml name=crowdsecurity/CVE-2022-35914 time="25-05-2023 14:17:44" level=info msg="Adding trigger bucket" cfg=ancient-feather file=/etc/crowdsec/scenarios/http-cve-2021-42013.yaml name=crowdsecurity/http-cve-2021-42013 time="25-05-2023 14:17:44" level=info msg="Adding leaky bucket" cfg=bold-tree file=/etc/crowdsec/scenarios/postfix-spam.yaml name=crowdsecurity/postfix-spam time="25-05-2023 14:17:44" level=info msg="Adding trigger bucket" cfg=holy-wave file=/etc/crowdsec/scenarios/postfix-spam.yaml name=crowdsecurity/postscreen-rbl time="25-05-2023 14:17:44" level=info msg="Adding trigger bucket" cfg=divine-bird file=/etc/crowdsec/scenarios/CVE-2019-18935.yaml name=crowdsecurity/CVE-2019-18935 time="25-05-2023 14:17:44" level=info msg="Adding trigger bucket" cfg=young-field file=/etc/crowdsec/scenarios/http-open-proxy.yaml name=crowdsecurity/http-open-proxy time="25-05-2023 14:17:44" level=info msg="Adding trigger bucket" cfg=fragrant-bush file=/etc/crowdsec/scenarios/grafana-cve-2021-43798.yaml name=crowdsecurity/grafana-cve-2021-43798 time="25-05-2023 14:17:44" level=info msg="Adding leaky bucket" cfg=proud-wood file=/etc/crowdsec/scenarios/http-generic-bf.yaml name=crowdsecurity/http-generic-bf time="25-05-2023 14:17:44" level=info msg="Adding leaky bucket" cfg=aged-star file=/etc/crowdsec/scenarios/http-generic-bf.yaml name=LePresidente/http-generic-401-bf time="25-05-2023 14:17:44" level=info msg="Adding leaky bucket" cfg=nameless-star file=/etc/crowdsec/scenarios/http-generic-bf.yaml name=LePresidente/http-generic-403-bf time="25-05-2023 14:17:44" level=info msg="Adding trigger bucket" cfg=holy-morning file=/etc/crowdsec/scenarios/zz-g00g1-dovecot.yaml name=g00g1/dovecot-bf time="25-05-2023 14:17:44" level=info msg="Adding trigger bucket" cfg=morning-firefly file=/etc/crowdsec/scenarios/fortinet-cve-2018-13379.yaml name=crowdsecurity/fortinet-cve-2018-13379 time="25-05-2023 14:17:44" level=info msg="Adding trigger bucket" cfg=twilight-river file=/etc/crowdsec/scenarios/http-cve-2021-41773.yaml name=crowdsecurity/http-cve-2021-41773 time="25-05-2023 14:17:44" level=warning msg="Loaded 49 scenarios" time="25-05-2023 14:17:44" level=info msg="Adding file /var/log/mail.log to filelist" type="file:///var/log/mail.log" time="25-05-2023 14:17:44" level=info msg="Starting processing data" time="25-05-2023 14:17:44" level=info msg="reading /var/log/mail.log at once" type="file:///var/log/mail.log" time="25-05-2023 14:17:44" level=warning msg="prometheus: listen tcp 127.0.0.1:6060: bind: address already in use" fatal error: concurrent map read and map write goroutine 13 [running]: github.com/crowdsecurity/crowdsec/pkg/parser.Parse({{_, _}, {_, _, _}, _, {_, _}}, {0x0, 0x1, ...}, ...) github.com/crowdsecurity/crowdsec/pkg/parser/runtime.go:329 +0x1128 main.runParse(0xc0023c48a0, 0xc0023c4840?, {{0xc002342300, 0x0}, {0xc000fcda40, 0x3, 0x4}, 0x1, {0xc001e79830, 0x16}}, ...) github.com/crowdsecurity/crowdsec/cmd/crowdsec/parse.go:33 +0x3a5 main.runCrowdsec.func1.1() github.com/crowdsecurity/crowdsec/cmd/crowdsec/crowdsec.go:57 +0xe8 gopkg.in/tomb%2ev2.(*Tomb).run(0x30c82a0, 0xc0023f6050?) gopkg.in/tomb.v2@v2.0.0-20161208151619-d5d1b5820637/tomb.go:163 +0x36 created by gopkg.in/tomb%2ev2.(*Tomb).Go gopkg.in/tomb.v2@v2.0.0-20161208151619-d5d1b5820637/tomb.go:159 +0xee goroutine 1 [chan receive]: main.Serve(0xc0023c64d0, 0x1?, 0x1e483e1?) github.com/crowdsecurity/crowdsec/cmd/crowdsec/serve.go:370 +0x7c5 main.StartRunSvc() github.com/crowdsecurity/crowdsec/cmd/crowdsec/run_in_svc.go:61 +0x3b5 main.main() github.com/crowdsecurity/crowdsec/cmd/crowdsec/main.go:336 +0x225 goroutine 22 [select]: database/sql.(*DB).connectionOpener(0xc002335d40, {0x225e030, 0xc0023f6000}) database/sql/sql.go:1218 +0x8d created by database/sql.OpenDB database/sql/sql.go:791 +0x18d goroutine 119 [select]: main.waitOnTomb() github.com/crowdsecurity/crowdsec/cmd/crowdsec/crowdsec.go:227 +0x7d main.serveCrowdsec.func1() github.com/crowdsecurity/crowdsec/cmd/crowdsec/crowdsec.go:150 +0xdd gopkg.in/tomb%2ev2.(*Tomb).run(0x30c81e0, 0xc002247780?) gopkg.in/tomb.v2@v2.0.0-20161208151619-d5d1b5820637/tomb.go:163 +0x36 created by gopkg.in/tomb%2ev2.(*Tomb).Go gopkg.in/tomb.v2@v2.0.0-20161208151619-d5d1b5820637/tomb.go:159 +0xee goroutine 134 [select]: net/http.(*persistConn).writeLoop(0xc001efefc0) net/http/transport.go:2410 +0xf2 created by net/http.(*Transport).dialConn net/http/transport.go:1766 +0x173d goroutine 133 [IO wait]: internal/poll.runtime_pollWait(0x7f3abc5b2110, 0x72) runtime/netpoll.go:306 +0x89 internal/poll.(*pollDesc).wait(0xc002570980?, 0xc000c96000?, 0x0) internal/poll/fd_poll_runtime.go:84 +0x32 internal/poll.(*pollDesc).waitRead(...) internal/poll/fd_poll_runtime.go:89 internal/poll.(*FD).Read(0xc002570980, {0xc000c96000, 0x1000, 0x1000}) internal/poll/fd_unix.go:167 +0x299 net.(*netFD).Read(0xc002570980, {0xc000c96000?, 0x419573?, 0x2000?}) net/fd_posix.go:55 +0x29 net.(*conn).Read(0xc000616280, {0xc000c96000?, 0x5?, 0xc0024214b0?}) net/net.go:183 +0x45 net/http.(*persistConn).Read(0xc001efefc0, {0xc000c96000?, 0x407926?, 0x60?}) net/http/transport.go:1943 +0x4e bufio.(*Reader).fill(0xc0015a8840) bufio/bufio.go:106 +0xff bufio.(*Reader).Peek(0xc0015a8840, 0x1) bufio/bufio.go:144 +0x5d net/http.(*persistConn).readLoop(0xc001efefc0) net/http/transport.go:2107 +0x1ac created by net/http.(*Transport).dialConn net/http/transport.go:1765 +0x16ea goroutine 15 [runnable]: reflect.Value.MapIndex({0x1bb64c0?, 0xc001e2aa20?, 0x15?}, {0x1b04e60, 0xc0012a1540, 0x98}) reflect/value.go:1723 +0x265 github.com/antonmedv/expr/vm/runtime.Fetch({0x1bb64c0, 0xc001e2aa20?}, {0x1b04e60?, 0xc0012a1540}) github.com/antonmedv/expr@v1.12.5/vm/runtime/runtime.go:55 +0x2e7 github.com/antonmedv/expr/vm.(*VM).Run(0xc0018d0328, 0xc001c79170, {0x1bb5c20?, 0xc00115aea0?}) github.com/antonmedv/expr@v1.12.5/vm/vm.go:120 +0x930 github.com/antonmedv/expr/vm.Run(0xc00243db36?, {0x1bb5c20?, 0xc00115aea0?}) github.com/antonmedv/expr@v1.12.5/vm/vm.go:27 +0x6a github.com/antonmedv/expr.Run(...) github.com/antonmedv/expr@v1.12.5/expr.go:185 github.com/crowdsecurity/crowdsec/pkg/parser.(*Node).process(0xc0018d1320, 0xc00134a000, {{0xc002342300, 0x0}, {0xc000fcda40, 0x3, 0x4}, 0x1, {0xc001e79830, 0x16}}, ...) github.com/crowdsecurity/crowdsec/pkg/parser/node.go:145 +0xd4 github.com/crowdsecurity/crowdsec/pkg/parser.Parse({{_, _}, {_, _, _}, _, {_, _}}, {0x0, 0x1, ...}, ...) github.com/crowdsecurity/crowdsec/pkg/parser/runtime.go:322 +0x102d main.runParse(0xc0023c48a0, 0xc0023c4840?, {{0xc002342300, 0x0}, {0xc000fcda40, 0x3, 0x4}, 0x1, {0xc001e79830, 0x16}}, ...) github.com/crowdsecurity/crowdsec/cmd/crowdsec/parse.go:33 +0x3a5 main.runCrowdsec.func1.1() github.com/crowdsecurity/crowdsec/cmd/crowdsec/crowdsec.go:57 +0xe8 gopkg.in/tomb%2ev2.(*Tomb).run(0x30c82a0, 0xc0023f6050?) gopkg.in/tomb.v2@v2.0.0-20161208151619-d5d1b5820637/tomb.go:163 +0x36 created by gopkg.in/tomb%2ev2.(*Tomb).Go gopkg.in/tomb.v2@v2.0.0-20161208151619-d5d1b5820637/tomb.go:159 +0xee goroutine 14 [runnable]: main.runParse(0xc0023c48a0, 0xc0023c4840?, {{0xc002342300, 0x0}, {0xc000fcda40, 0x3, 0x4}, 0x1, {0xc001e79830, 0x16}}, ...) github.com/crowdsecurity/crowdsec/cmd/crowdsec/parse.go:40 +0x805 main.runCrowdsec.func1.1() github.com/crowdsecurity/crowdsec/cmd/crowdsec/crowdsec.go:57 +0xe8 gopkg.in/tomb%2ev2.(*Tomb).run(0x30c82a0, 0xc0023f6050?) gopkg.in/tomb.v2@v2.0.0-20161208151619-d5d1b5820637/tomb.go:163 +0x36 created by gopkg.in/tomb%2ev2.(*Tomb).Go gopkg.in/tomb.v2@v2.0.0-20161208151619-d5d1b5820637/tomb.go:159 +0xee goroutine 120 [chan receive]: gopkg.in/tomb%2ev2.(*Tomb).Wait(0x30c80c0) gopkg.in/tomb.v2@v2.0.0-20161208151619-d5d1b5820637/tomb.go:126 +0x35 github.com/crowdsecurity/crowdsec/pkg/acquisition.StartAcquisition({0xc00140c950, 0x1, 0xc000087f30?}, 0xc0023c48a0, 0x30c80c0) github.com/crowdsecurity/crowdsec/pkg/acquisition/acquisition.go:332 +0x1ed main.runCrowdsec(0xc0023c64d0, 0xc001b92080) github.com/crowdsecurity/crowdsec/cmd/crowdsec/crowdsec.go:125 +0x32d main.serveCrowdsec.func1.1() github.com/crowdsecurity/crowdsec/cmd/crowdsec/crowdsec.go:141 +0xe5 created by main.serveCrowdsec.func1 github.com/crowdsecurity/crowdsec/cmd/crowdsec/crowdsec.go:136 +0xd8 goroutine 16 [select]: main.runPour(0xc0023c4840, {0xc001790000, 0x31, 0x49}, 0xc0013df050?, 0xc0023c64d0) github.com/crowdsecurity/crowdsec/cmd/crowdsec/pour.go:20 +0xfd main.runCrowdsec.func2.1() github.com/crowdsecurity/crowdsec/cmd/crowdsec/crowdsec.go:83 +0x6c gopkg.in/tomb%2ev2.(*Tomb).run(0x30c8180, 0x0?) gopkg.in/tomb.v2@v2.0.0-20161208151619-d5d1b5820637/tomb.go:163 +0x36 created by gopkg.in/tomb%2ev2.(*Tomb).Go gopkg.in/tomb.v2@v2.0.0-20161208151619-d5d1b5820637/tomb.go:159 +0xee goroutine 130 [select]: net/http.(*persistConn).roundTrip(0xc001efefc0, 0xc000e14100) net/http/transport.go:2638 +0x994 net/http.(*Transport).roundTrip(0x2faa240, 0xc000f38400) net/http/transport.go:603 +0x7fa net/http.(*Transport).RoundTrip(0xc000f38300?, 0x40fb6a?) net/http/roundtrip.go:17 +0x19 github.com/crowdsecurity/crowdsec/pkg/apiclient.retryRoundTripper.RoundTrip({{0x224d140, 0x2faa240}, 0x5, {0xc0003760f0, 0x3, 0x3}, 0x1, 0x0}, 0xc000f38300) github.com/crowdsecurity/crowdsec/pkg/apiclient/auth.go:118 +0x2ae github.com/crowdsecurity/crowdsec/pkg/apiclient.retryRoundTripper.RoundTrip({{0x2249e40, 0xc000e14080}, 0x2, {0xc0001b85b0, 0x2, 0x2}, 0x0, 0xc000f4a1d0}, 0xc000f38200) github.com/crowdsecurity/crowdsec/pkg/apiclient/auth.go:118 +0x2ae github.com/crowdsecurity/crowdsec/pkg/apiclient.(*JWTTransport).RoundTrip(0xc0013da000, 0xc000f38200) github.com/crowdsecurity/crowdsec/pkg/apiclient/auth.go:258 +0x54d net/http.send(0xc000f38200, {0x2249e20, 0xc0013da000}, {0x8?, 0x1de4bc0?, 0x0?}) net/http/client.go:252 +0x5f7 net/http.(*Client).send(0xc0001d4060, 0xc000f38200, {0x4053d4?, 0x1d610f8?, 0x0?}) net/http/client.go:176 +0x9b net/http.(*Client).do(0xc0001d4060, 0xc000f38200) net/http/client.go:716 +0x8fb net/http.(*Client).Do(...) net/http/client.go:582 github.com/crowdsecurity/crowdsec/pkg/apiclient.(*ApiClient).Do(0xc002570080, {0x225e068, 0xc0000460b0}, 0xc000f38100, {0x1c4a0a0, 0xc0001d4150}) github.com/crowdsecurity/crowdsec/pkg/apiclient/client_http.go:65 +0x325 github.com/crowdsecurity/crowdsec/pkg/apiclient.(*AuthService).AuthenticateWatcher(0xc002570088, {0x225e068, 0xc0000460b0}, {0xc000f86170, 0xc000f4a050, {0xc000b0c800, 0x2a, 0x40}}) github.com/crowdsecurity/crowdsec/pkg/apiclient/auth_service.go:63 +0x1a5 main.runOutput(0xc0023c4840?, 0xc001c7b980, 0xc0013df050, {{0xc00143a8d0, 0x0}, {0x0, 0x0, 0x0}, 0x1, {0xc001e79830, ...}}, ...) github.com/crowdsecurity/crowdsec/cmd/crowdsec/output.go:100 +0x68e main.runCrowdsec.func3.1() github.com/crowdsecurity/crowdsec/cmd/crowdsec/crowdsec.go:101 +0x166 gopkg.in/tomb%2ev2.(*Tomb).run(0x30c8240, 0x0?) gopkg.in/tomb.v2@v2.0.0-20161208151619-d5d1b5820637/tomb.go:163 +0x36 created by gopkg.in/tomb%2ev2.(*Tomb).Go gopkg.in/tomb.v2@v2.0.0-20161208151619-d5d1b5820637/tomb.go:159 +0xee goroutine 125 [runnable]: github.com/crowdsecurity/crowdsec/pkg/acquisition/modules/file.(*FileSource).readFile(0xc001c68d80, {0x7ffdd9b146a0, 0x11}, 0x0?, 0x30c80c0) github.com/crowdsecurity/crowdsec/pkg/acquisition/modules/file/file.go:538 +0x8e8 github.com/crowdsecurity/crowdsec/pkg/acquisition/modules/file.(*FileSource).OneShotAcquisition(0xc001c68d80, 0x24?, 0xc001ac0ba0?) github.com/crowdsecurity/crowdsec/pkg/acquisition/modules/file/file.go:267 +0x210 github.com/crowdsecurity/crowdsec/pkg/acquisition.StartAcquisition.func1() github.com/crowdsecurity/crowdsec/pkg/acquisition/acquisition.go:320 +0x43b gopkg.in/tomb%2ev2.(*Tomb).run(0x30c80c0, 0xc001c8e240?) gopkg.in/tomb.v2@v2.0.0-20161208151619-d5d1b5820637/tomb.go:163 +0x36 created by gopkg.in/tomb%2ev2.(*Tomb).Go gopkg.in/tomb.v2@v2.0.0-20161208151619-d5d1b5820637/tomb.go:159 +0xee goroutine 126 [select]: github.com/crowdsecurity/crowdsec/pkg/leakybucket.LeakRoutine(0xc0022b4000) github.com/crowdsecurity/crowdsec/pkg/leakybucket/bucket.go:234 +0x9ff github.com/crowdsecurity/crowdsec/pkg/leakybucket.LoadOrStoreBucketFromHolder.func1() github.com/crowdsecurity/crowdsec/pkg/leakybucket/manager_run.go:269 +0x1d gopkg.in/tomb%2ev2.(*Tomb).run(0x30c8180, 0xc001c8e240?) gopkg.in/tomb.v2@v2.0.0-20161208151619-d5d1b5820637/tomb.go:163 +0x36 created by gopkg.in/tomb%2ev2.(*Tomb).Go gopkg.in/tomb.v2@v2.0.0-20161208151619-d5d1b5820637/tomb.go:159 +0xee FATA[25-05-2023 14:17:44] fail to run crowdsec for test: exit status 2 ```

LaurenceJJones commented 1 year ago

Okay I will have to find some time to replicate. Sorry to point this out Gentoo is not officially supported by us so we cannot guarantee any version of CrowdSec operates on it cause we have no tests to validate this.

blotus commented 1 year ago

Hello @g00g1,

I think I found the cause of this issue.

While we are working on a fix, you can workaround this by setting parser_routines to 1 in your config and restarting crowdsec (note this will slow down a bit the parsing, which might or might not be acceptable in your case)

LaurenceJJones commented 1 year ago

I thought @buixor had already pushed a fix for this? With the mutex locks?