Closed AlexisPPLIN closed 1 year ago
github-actions[bot]
commented
1 year ago @AlexisPPLIN: Thanks for opening an issue, it is currently awaiting triage.
In the meantime, you can:
LaurenceJJones
commented
1 year ago Hey thank you for opening an issue:
The mount point you provided wouldn't make a direct difference because all parsers/scenarios are held under /etc/crowdsec/ rather than the data directory.
So the parsers do no work out of the box.
This is not the case, the term tainted means the local copy you have is not the same as the copy we have within the hub repo.
Tried to replicate but to no success maybe you have /etc/crowdsec/ mounted and it was removed on host?
vagrant@bullseye:/opt/cs-firewall-bouncer$ sudo docker run -d -e COLLECTIONS="crowdsecurity/traefik crowdsecurity/apache2 crowdsecurity/base-http-scenarios crowdsecurity/mariadb crowdsecurity/whitelist-good-actors" crowdsecurity/crowdsec:v1.4.6
9cb6872a18d78cf9608732a7c6d1cbb162cdbf4e4b31cbc0c837468c7fb62ab9
vagrant@bullseye:/opt/cs-firewall-bouncer$ sudo docker exec -it 9cb6872a18d78cf9608732a7c6d1cbb162cdbf4e4b31cbc0c837468c7fb62ab9 cscli collections list
COLLECTIONS
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Name 📦 Status Version Local Path
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────
crowdsecurity/apache2 ✔ enabled 0.1 /etc/crowdsec/collections/apache2.yaml
crowdsecurity/base-http-scenarios ✔ enabled 0.6 /etc/crowdsec/collections/base-http-scenarios.yaml
crowdsecurity/http-cve ✔ enabled 2.0 /etc/crowdsec/collections/http-cve.yaml
crowdsecurity/linux ✔ enabled 0.2 /etc/crowdsec/collections/linux.yaml
crowdsecurity/mariadb ✔ enabled 0.1 /etc/crowdsec/collections/mariadb.yaml
crowdsecurity/sshd ✔ enabled 0.2 /etc/crowdsec/collections/sshd.yaml
crowdsecurity/traefik ✔ enabled 0.1 /etc/crowdsec/collections/traefik.yaml
crowdsecurity/whitelist-good-actors ✔ enabled 0.1 /etc/crowdsec/collections/whitelist-good-actors.yaml
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────
vagrant@bullseye:/opt/cs-firewall-bouncer$Incase the mounting made a difference here a full reproduce:
vagrant@bullseye:~$ sudo docker run -d -v "$PWD/empty:/var/lib/crowdsec/data" -e COLLECTIONS="crowdsecurity/traefik crowdsecurity/apache2 crowdsecurity/base-http-scenarios crowdsecurity/mariadb crowdsecurity/whitelist-good-actors" crowdsecurity/crowdsec:v1.4.6
1e8c3b94c531cfb1a2b8c4cb1cbf4b2df6b9c2caa696c0599d98e1b538e2d748
vagrant@bullseye:~$ ll
-bash: ll: command not found
vagrant@bullseye:~$ cd empty
vagrant@bullseye:~/empty$ ls -la
total 128
drwxr-xr-x 2 vagrant vagrant 4096 May 16 14:35 .
drwxr-xr-x 6 vagrant vagrant 4096 May 16 14:34 ..
lrwxrwxrwx 1 root root 48 May 16 14:35 GeoLite2-ASN.mmdb -> /staging/var/lib/crowdsec/data/GeoLite2-ASN.mmdb
lrwxrwxrwx 1 root root 49 May 16 14:35 GeoLite2-City.mmdb -> /staging/var/lib/crowdsec/data/GeoLite2-City.mmdb
-rw-r--r-- 1 root root 2593 May 16 14:35 backdoors.txt
-rw-r--r-- 1 root root 9824 May 16 14:35 bad_user_agents.regex.txt
-rw-r--r-- 1 root root 104 May 16 14:35 cloudflare_ip6s.txt
-rw-r--r-- 1 root root 230 May 16 14:35 cloudflare_ips.txt
-rw-r----- 1 root root 57344 May 16 14:35 crowdsec.db
-rw-r--r-- 1 root root 448 May 16 14:35 http_path_traversal.txt
-rw-r--r-- 1 root root 401 May 16 14:35 ip_seo_bots.txt
-rw-r--r-- 1 root root 749 May 16 14:35 jira_cve_2021-26086.txt
-rw-r--r-- 1 root root 1432 May 16 14:35 log4j2_cve_2021_44228.txt
-rw-r--r-- 1 root root 245 May 16 14:35 rdns_seo_bots.regex
-rw-r--r-- 1 root root 154 May 16 14:35 rdns_seo_bots.txt
-rw-r--r-- 1 root root 1021 May 16 14:35 sensitive_data.txt
-rw-r--r-- 1 root root 249 May 16 14:35 sqli_probe_patterns.txt
-rw-r--r-- 1 root root 935 May 16 14:35 thinkphp_cve_2018-20062.txt
-rw-r--r-- 1 root root 274 May 16 14:35 xss_probe_patterns.txt
vagrant@bullseye:~/empty$ sudo docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
1e8c3b94c531 crowdsecurity/crowdsec:v1.4.6 "/bin/sh -c '/bin/ba…" 20 seconds ago Up 19 seconds nifty_dubinsky
vagrant@bullseye:~/empty$ sudo docker exec -it 1e8c3b94c531 cscli collections list
COLLECTIONS
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Name 📦 Status Version Local Path
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────
crowdsecurity/apache2 ✔ enabled 0.1 /etc/crowdsec/collections/apache2.yaml
crowdsecurity/base-http-scenarios ✔ enabled 0.6 /etc/crowdsec/collections/base-http-scenarios.yaml
crowdsecurity/http-cve ✔ enabled 2.0 /etc/crowdsec/collections/http-cve.yaml
crowdsecurity/linux ✔ enabled 0.2 /etc/crowdsec/collections/linux.yaml
crowdsecurity/mariadb ✔ enabled 0.1 /etc/crowdsec/collections/mariadb.yaml
crowdsecurity/sshd ✔ enabled 0.2 /etc/crowdsec/collections/sshd.yaml
crowdsecurity/traefik ✔ enabled 0.1 /etc/crowdsec/collections/traefik.yaml
crowdsecurity/whitelist-good-actors ✔ enabled 0.1 /etc/crowdsec/collections/whitelist-good-actors.yaml
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────
vagrant@bullseye:~/empty$
AlexisPPLIN
commented
1 year ago Thanks for your reply !
This is not the case, the term tainted means the local copy you have is not the same as the copy we have within the hub repo.
My bad, thanks for the clarification.
Tried to replicate but to no success maybe you have /etc/crowdsec/ mounted and it was removed on host?
I only have two mount on this container : .docker_data/crowdsec:/var/lib/crowdsec/data and /var/run/docker.sock:/var/run/docker.sock
$ ls -l .docker_data/crowdsec
-rw-r--r--. 1 root root 2593 16 mai 16:02 backdoors.txt
-rw-r--r--. 1 root root 9824 16 mai 16:02 bad_user_agents.regex.txt
-rw-r--r--. 1 root root 104 16 mai 16:02 cloudflare_ip6s.txt
-rw-r--r--. 1 root root 230 16 mai 16:02 cloudflare_ips.txt
-rw-r-----. 1 root root 57344 16 mai 16:26 crowdsec.db
lrwxrwxrwx. 1 root root 48 16 mai 16:02 GeoLite2-ASN.mmdb -> /staging/var/lib/crowdsec/data/GeoLite2-ASN.mmdb
lrwxrwxrwx. 1 root root 49 16 mai 16:02 GeoLite2-City.mmdb -> /staging/var/lib/crowdsec/data/GeoLite2-City.mmdb
-rw-r--r--. 1 root root 448 16 mai 16:02 http_path_traversal.txt
-rw-r--r--. 1 root root 401 16 mai 16:02 ip_seo_bots.txt
-rw-r--r--. 1 root root 749 16 mai 16:02 jira_cve_2021-26086.txt
-rw-r--r--. 1 root root 1432 16 mai 16:02 log4j2_cve_2021_44228.txt
-rw-r--r--. 1 root root 245 16 mai 16:02 rdns_seo_bots.regex
-rw-r--r--. 1 root root 154 16 mai 16:02 rdns_seo_bots.txt
-rw-r--r--. 1 root root 1021 16 mai 16:02 sensitive_data.txt
-rw-r--r--. 1 root root 249 16 mai 16:02 sqli_probe_patterns.txt
-rw-r--r--. 1 root root 935 16 mai 16:02 thinkphp_cve_2018-20062.txt
-rw-r--r--. 1 root root 274 16 mai 16:02 xss_probe_patterns.txtMaybe it's a misconfig in my Dockerfile ?
FROM crowdsecurity/crowdsec:v1.4.6
COPY ./docker/config/crowdsec/acquis.yaml /staging/etc/crowdsec/acquis.yaml
COPY ./docker/config/crowdsec/whitelist.yaml /etc/crowdsec/parsers/s02-enrich/whitelist.yaml
ENV TZ=Europe/Paris
ENV COLLECTIONS="crowdsecurity/traefik crowdsecurity/apache2 crowdsecurity/base-http-scenarios crowdsecurity/mariadb crowdsecurity/whitelist-good-actors"
HEALTHCHECK --interval=5s --timeout=5s --start-period=5s --retries=5 \
CMD wget -nv -t1 --spider http://localhost:8080/healthI tried to replicate it to no success even using a custom image like the one you provided. So I am going to close the issue as completed, if the issue comes back or persists please reopen
What happened?
Hi !
With the last version of the crowdsec docker image (
crowdsecurity/crowdsec:v1.4.6). By default the collectioncrowdsecurity/linuxandcrowdsecurity/sshdare tainted.So the parsers do no work out of the box.
Did I miss something ?
What did you expect to happen?
On a new container, using
COLLECTIONenvironnement variable. Every collections should be working without others commands.Workaround
After the first start, we have to reinstall the
crowdsecurity/linuxcollection.How can we reproduce it (as minimally and precisely as possible)?
1. Start fresh new
crowdsecurity/crowdsecdocker container :/var/lib/crowdsec/dataempty.COLLECTIONS="crowdsecurity/traefik crowdsecurity/apache2 crowdsecurity/base-http-scenarios crowdsecurity/mariadb crowdsecurity/whitelist-good-actors"2. Attach with bash on newly started container :
3. Check installed collections :
As you see above,
crowdsecurity/linuxandcrowdsecurity/sshdis marked as⚠️ enabled,taintedAnything else we need to know?
Crowdsec logs :
Crowdsec version
OS version
Enabled collections and parsers
Acquisition config
Config show
Prometheus metrics
Related custom configs versions (if applicable) : notification plugins, custom scenarios, parsers etc.