Improvement/Ispconfig vhosts apache logs

crowdsecurity / crowdsec

CrowdSec - the open-source and participative security solution offering crowdsourced protection against malicious IPs and access to the most advanced real-world CTI.

https://crowdsec.net

MIT License

8.58k stars 437 forks source link

Improvement/Ispconfig vhosts apache logs #574

Open lukyrys opened 3 years ago

lukyrys commented 3 years ago

Is possible add detecting ispconfig apache logs? in default ispconfig does not merge all vhosts to default access.log or other_vhosts_access.log under /var/log/apache2 Per webiste apache logs is placed under /var/log/ispconfig/httpd/< website vhost >/access.log ( /var/log/ispconfig/httpd/*/access.log ) access.log is symlink to actual date log in format 20210115-access.log

+ i noticed false detect ispconfig auth log placed at /var/log/ispconfig/auth.log for sshd auth.log

buixor commented 3 years ago

Are you refering to this : https://www.ispconfig.org/ ?

lukyrys commented 3 years ago

@buixor yes

buixor commented 3 years ago

Hello @lukyrys !

@sbs2001 opened an MR regarding this, but needs some testing on your side, can you help please ?

Thanks !

LaurenceJJones commented 2 years ago

Hey wanted to get involved in another good first issue!

Looking into the MR that was opened it would of worked but it seems that find with the -name parameter does not accept wildcard in the path variable.

So the way the wizard is setup even this should work since find is recursive by default

/var/log/ispconfig/httpd/access.log

LaurenceJJones commented 1 year ago

So coming back after failing the first time.

The issue is, the latest access.log for lspconfig is a symlink to the latest log file, so wizard.sh will always miss it. However @mmetc has been working on an improvement setup tool that can detect running services. So we can detect apache2 running in lspconfig the user would just need to edit the path.

mmetc commented 1 year ago

I have not followed until now, but I am not sure service detection has anything to do with this - we already know that the service is there. The issue is locating the log files.

There are several sub-issues, correct me if I'm wrong:

latest.log is a symlink, when it rotates crowdsec is confused. Can we confirm this?
we can use globbing to read the yyyymmdd-access.log files, but afaik it only supports one level (i.e. no /var/log/ispconfig/httpd/*/*.log ). Can we confirm this?
if we catch all .log files, we will have both latest.log and the file that latest.log links to -- hence we process stuff twice and the scenarios are triggered more easily. We can easily avoid that with an exclude regexp (from v 1.4.2) or by using -access.log

ThomasCr commented 1 year ago

Any Update or Workaround?

mmetc commented 1 year ago

Hi @ThomasCr , by workaround you mean it doesn't work even with a manual setup?

Can you try this: in /etc/crowdsec/acquis.d/ispconfig.yaml

filenames:
  - /var/log/ispconfig/httpd/*/access.log
labels:
  type: apache2

then add this to /etc/crowdsec/config.yaml

crowdsec_service:
    acquisition_dir: /etc/crowdsec/acquis.d

Symlinks and double globbing are supported, so even /var/log/ispconfig/httpd//-access.log should work.

If your issue is automating the configuration, the current wizard.sh script is basically frozen. We are testing a new functionality in 1.5.0 but it's not ready yet. Adding the above file is a noop if the software is not there, so it could be enough for you.

LaurenceJJones commented 1 year ago

latest.log is a symlink, when it rotates crowdsec is confused. Can we confirm this?

Yes crowdsec doesn't know symlink has been updated so doesn't follow any updates

kepon85 commented 2 months ago

I use Crowdsec and ISPconfig, here's how I did it: https://david.mercereau.info/crowdsec-ispconfig-iptables/ it's a bit of DIY but it works.