search

crowdsecurity / crowdsec

CrowdSec - the open-source and participative security solution offering crowdsourced protection against malicious IPs and access to the most advanced real-world CTI.
https://crowdsec.net
MIT License
8.98k stars 466 forks source link

Bug/Upgrade from crowdsec 1.1.1 -> 1.2.1 fails on Debian #1111

Closed klausagnoletti closed 2 years ago

klausagnoletti commented 2 years ago

Describe the bug On a clean install of Debian 10.11 I first installed crowdsec 1.1.1 and iptables bouncer 0.0.15 from debs. Next I upgraded. That failed because crowdsec failed to start after upgrade. 

time="17-12-2021 17:27:22" level=error msg="failed to load pattern mysql.dpkg-dist : the pattern already exist"
time="17-12-2021 17:27:22" level=fatal msg="crowdsec init: Failed to load parsers: failed to load parser patterns : the pattern already exist"

This was in the logs. After deleting the mysql.dpkg-dist in /etc/crowdsec/patterns I was able to start the crowdsec agent and finish the upgrade.

To Reproduce Steps to reproduce the behavior:

  1. Install Debian 10.11
  2. Add package cloud repo
  3. install crowdsec=1.1.1 and crowdsec-firewall-iptables=0.1.5
  4. apt upgrade so latest version of both packages are installed

Expected behavior I expected upgrade to not fail :-)

Technical Information (please complete the following information):

thib3113 commented 2 years ago

Go in /etc/crowdsec/parsers/, you will maybe got some patterns in double (and some with .dpkg-dist in the name)

interzone2012 commented 2 years ago

Op vr 17 dec. 2021 21:44 schreef Thibaut SEVERAC @.***>:

Go in /etc/crowdsec/parsers/, you will maybe got some patterns in double (and some with .dpkg-dist in the name)

— Reply to this email directly, view it on GitHub https://github.com/crowdsecurity/crowdsec/issues/1111#issuecomment-997020433, or unsubscribe https://github.com/notifications/unsubscribe-auth/APEURA5DL3ETOFZG2JBGHK3UROOJLANCNFSM5KJLS2QA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you are subscribed to this thread.Message ID: @.***>

Athanasius commented 2 years ago

I too encountered this. /etc/crowdsec/parsers contained no duplicates (checked with ls -alR in there).

TL;DR - Likely something to do with leapfrogging so many versions or the .dpkg file not ensuring it definitely updates all the relevant files to their latest versions.

In the end I:

  1. backed up the two credentials files (I had no custom config, and all of /etc is under git anyway).
  2. apt purge crowdsec
  3. apt install crowdsec (which made new credentials files anyway)

and it worked. Now, I was coming from some ancient 1.0.2 version, because no hint was given that the APT repository had changed, so it's possible that necessary tweaks to configuration/other files were missed due to that leap.

I note that performing that upgrade did not change the files under /etc/crowdsec/parsers (which are of course symlinks) or the files those symlinks point to, but the purge/install cycle has. So something definitely went awry with the APT upgrade of the package.

I had /etc/ committed and in a clean state before the purge/install, but after the upgrade from 1.0.2 to 1.3.1. Now I've done the purge/install cycle I'm showing differences in, e.g. crowdsec/hub/parsers/s00-raw/crowdsecurity/syslog-logs.yaml in a git diff. I've now commited those changes, and for my own reference that's local /etc commit c165f4aea87d68f530dad5f285e85d5aa6bfba8.

sabban commented 2 years ago

Hi,

I believe this commit https://github.com/crowdsecurity/crowdsec/commit/1f06f242cc18ca99cc9f69164341592139f85d64 fixes this issue.

Regards,