search

mitchellkrogza / Fail2Ban-Blacklist-JAIL-for-Repeat-Offenders-with-Perma-Extended-Banning

A customised jail with action and filter file for Fail2Ban. This jail is based on the recidive jail but makes use of a simple text file to enable extended and permanent bans.
150 stars 37 forks source link

error-log or access-log for [apache-noscript]? #12

Open robert1112 opened 5 years ago

robert1112 commented 5 years ago

Hi

I am not sure if it is correct place to post but this is the best I can come up with. Sorry if it is inappropriate.

I follow your post here https://ubuntu101.co.za/security/fail2ban/fail2ban-persistent-bans- and left a comment but I think you haven't seen it yet. The main point is I got the recommendation from fail2ban that it is supposed to be logpath = %(apache_error_log)s in [apache-noscript] config file.

Here is the link to the issue I posted. https://github.com/fail2ban/fail2ban/issues/2344

Can you kindly help? Thank you so much.

mitchellkrogza commented 5 years ago

Hi @robert1112 yes you should use logpath = %(apache_error_log)s if that does not work then your main fail2ban config is not setup properly.

Check how your paths have been specified in either /etc/fail2ban/paths-common.conf or /etc/fail2ban/paths-debian.conf depending which one you have specified in your jail.conf or jail.local in the section 

[INCLUDES]

before = paths-debian.conf
robert1112 commented 5 years ago

Hi @mitchellkrogza Thank you so much. I will take a closer look at your suggestion. In addition, should I change all to error-log from access-log? Thank you so much.

[apache-auth]
enabled = true
port     = http,https
logpath  = /var/log/apache2/*access.log*

[apache-badbots]
# Ban hosts which agent identifies spammer robots crawling the web
# for email addresses. The mail outputs are buffered.
enabled = true
port     = http,https
logpath  = /var/log/apache2/*access.log*
bantime  = 172800
maxretry = 1

[apache-noscript]
enabled = true
port     = http,https
logpath  = /var/log/apache2/*access.log*
maxretry = 6

[apache-botsearch]
enabled = true
port     = http,https
logpath  = /var/log/apache2/*access.log*
maxretry = 2
mitchellkrogza commented 5 years ago

No don't change them all to error.log .... some jails require reading an error.log file but 99% of jails require reading an access.log file.

robert1112 commented 5 years ago

I see. What about the regex? How should I change the code below? I am sorry I am not a programmer so it is a bit difficult for me to tell what is it. Thank you.

failregex = ^%(_apache_error_client)s ((AH001(28|30): )?File does not exist|(AH01264: )?script not found or unable to stat): /\S*(php([45]|[.-]cgi)?|\.asp|\.exe|\.pl)(, referer: \S+)?\s*$
            ^%(_apache_error_client)s script '/\S*(php([45]|[.-]cgi)?|\.asp|\.exe|\.pl)\S*' not found or unable to stat(, referer: \S+)?\s*$
failregex = ^<HOST> [^"]*"[A-Z]+ /\S+(?:\.(?:asp|exe|pl|cgi))\b[^"]*" [35]0\d\s
mitchellkrogza commented 5 years ago

What filter is that and why are you trying to change it from the defaults?

robert1112 commented 5 years ago

Hi It is /etc/fail2ban/filter.d/apache-noscript.conf. I use Wordpress so I want to remove php from the regex. Thank you for your support.

# Fail2Ban filter to block web requests for scripts (on non scripted websites)
#
# This matches many types of scripts that don't exist. This could generate a
# lot of false positive matches in cases like wikis and forums where users
# no affiliated with the website can insert links to missing files/scripts into
# pages and cause non-malicious browsers of the site to trigger against this
# filter.
#
# If you'd like to match specific URLs that don't exist see the
# apache-botsearch filter.
#

[INCLUDES]

# overwrite with apache-common.local if _apache_error_client is incorrect.
before = apache-common.conf

[Definition]

failregex = ^%(_apache_error_client)s ((AH001(28|30): )?File does not exist|(AH01264: )?script not found or unable to stat): /\S*(php([45]|[.-]cgi)?|\.asp|\.exe|\.pl)(, referer: \S+)?\s*$
            ^%(_apache_error_client)s script '/\S*(php([45]|[.-]cgi)?|\.asp|\.exe|\.pl)\S*' not found or unable to stat(, referer: \S+)?\s*$

ignoreregex = 

# DEV Notes:
#
# https://wiki.apache.org/httpd/ListOfErrors for apache error IDs
#
# Second regex, script '/\S*(\.php|\.asp|\.exe|\.pl)\S*' not found or unable to stat\s*$ is in httpd-2.2
#
# Author: Cyril Jaquier
mitchellkrogza commented 5 years ago

Try this ... BUT test it thoroughly on a live site.

[Definition]

failregex = ^%(_apache_error_client)s ((AH001(28|30): )?File does not exist|(AH01264: )?script not found or unable to stat): /\S*([.-]cgi)?|\.asp|\.exe|\.pl)(, referer: \S+)?\s*$
            ^%(_apache_error_client)s script '/\S*([.-]cgi)?|\.asp|\.exe|\.pl)\S*' not found or unable to stat(, referer: \S+)?\s*$
robert1112 commented 5 years ago

Thank you so much. I will update on this today or tomorrow. Thank you so much. 👍

mitchellkrogza commented 5 years ago

Make sure to reload fail2ban after that change so the new filter change is loaded.

robert1112 commented 5 years ago

Hi @mitchellkrogza

It is weird. Nothing popped up on my error.log while I run https://myip/test.asp/ from Safari Browser. I also created an attack server and run curl https://myip/test.asp/

Did I misunderstand something from your post?

This is my 2nd time leaning firewall, so maybe some basic concept is missing here. Please kindly help.😞

/etc/fail2ban$ cat paths-debian.conf

# Debian
[INCLUDES]
before = paths-common.conf
after  = paths-overrides.local
[DEFAULT]
syslog_mail = /var/log/mail.log
# control the `mail.warn` setting, see `/etc/rsyslog.d/50-default.conf` (if commented `mail.*` wins).
# syslog_mail_warn = /var/log/mail.warn
syslog_mail_warn = %(syslog_mail)s
syslog_user  =  /var/log/user.log
syslog_ftp = /var/log/syslog
syslog_daemon =  /var/log/daemon.log
exim_main_log = /var/log/exim4/mainlog
# was in debian squeezy but not in wheezy
# /etc/proftpd/proftpd.conf (SystemLog)
proftpd_log = /var/log/proftpd/proftpd.log
mitchellkrogza commented 5 years ago

Post your jail settings for this filter. How many times did you simulate the attack?

robert1112 commented 5 years ago

Hi @mitchellkrogza Sorry for my late reply. Here it comes. Thank you so much.

/etc/fail2ban$ cat jail.local

#
# Local Jail.conf File
# Comments: use '#' for comment lines and ';' (following a space) for inline comments

[INCLUDES]
before = paths-debian.conf

[DEFAULT]

# Add any IP's to ignore below - all on one line with spaces
# between them remove 111.111.111.111 and 222.222.222.222
# they are just here to demonstrate syntax
ignoreip = 127.0.0.1/8 106.105.213.54 121.254.111.144
ignorecommand =

# Ban and Fine Time in Seconds
bantime  = 600
findtime  = 600

# Maximum attempts before banning intruder
maxretry = 6

backend = auto
usedns = warn
logencoding = auto

# Default Action All Filters Disabled
enabled = false

# Default Filter Name Uses Jail Name
filter = %(__name__)s[mode=%(mode)s]

# Mail Settings
destemail = mymail
sender = mymail2
sendername = Fail2Ban
mta = mail

# Firewall Defaults
protocol = tcp
chain = <known/chain>
port = 0:65535
fail2ban_agent = Fail2Ban/%(fail2ban_version)s
banaction = iptables-multiport
banaction_allports = iptables-allports

# Our Banning Action
# ban & send an e-mail with whois report and relevant log lines
# to the destemail.
action_mwl = %(banaction)s[name=%(__name__)s, bantime="%(bantime)s", port="%(port)s", protocol="%(protocol)s", chain="%(chain)s"]
             %(mta)s-whois-lines[name=%(__name__)s, sender="%(sender)s", dest="%(destemail)s", logpath=%(logpath)s, chain="%(chain)s"]

# ban IP on CloudFlare & send an e-mail with whois report and relevant log lines
# to the destemail.
action_cf_mwl = cloudflare[cfuser="%(cfemail)s", cftoken="%(cfapikey)s"]
                %(mta)s-whois-lines[name=%(__name__)s, sender="%(sender)s", dest="%(destemail)s", logpath=%(logpath)s, chain="%(chain)s"]

# Choose default action.
action = %(action_mwl)s

# NOTE: Other actions removed. Review jail.conf file for all
# other available options like action_ action_mw action_xarf
# action_cf_mwl action_blocklist_de and action_badips
# I find action_mwl to be more than adequate for my needs and
# the others especially xarf, blocklist_de and badips should
# be used with utmost care and only when you know what you are doing

#
# JAILS
#

#
# SSH servers
#

[sshd]
enabled = true
port    = ssh
filter = sshd
logpath  = /var/log/auth.*
maxretry = 6

[sshd-ddos]
enabled = true
port    = ssh
filter = sshd-ddos
logpath  = /var/log/auth.*
maxretry = 6

#
# HTTP servers
#

[apache-auth]
enabled = true
port     = http,https
logpath  = /ebs/containers/blog6/ddg6gd/*access.log*

[apache-badbots]
# Ban hosts which agent identifies spammer robots crawling the web
# for email addresses. The mail outputs are buffered.
enabled = true
port     = http,https
logpath  = /ebs/containers/blog6/ddg6gd/*access.log*
bantime  = 172800
maxretry = 1

[apache-noscript]
enabled = true
port     = http,https
logpath  = /ebs/containers/blog6/ddg6gd/*error.log*
maxretry = 6

[apache-botsearch]
enabled = true
port     = http,https
logpath  = /ebs/containers/blog6/ddg6gd/*access.log*
maxretry = 2

# Jail for more extended banning of persistent abusers
# !!! WARNINGS !!!
# 1. Make sure that your loglevel specified in fail2ban.conf/.local
#    is not at DEBUG level -- which might then cause fail2ban to fall into
#    an infinite loop constantly feeding itself with non-informative lines
# 2. Increase dbpurgeage defined in fail2ban.conf to e.g. 648000 (7.5 days)
#    to maintain entries for failed logins for sufficient amount of time

[blacklist]
enabled = true
logpath  = /var/log/fail2ban.*
filter = blacklist
banaction = blacklist
action = %(action_)s
bantime  = 31536000   ; 1 year
findtime = 31536000   ; 1 year
maxretry = 10
robert1112 commented 5 years ago

Hi @mitchellkrogza Can you kindly suggest? I think I run more than 6 times. If I am correct, each run will generate error log in the error log? Or it will only generate an error log when up to 6 times? Either way, Fail2Ban doesn't take action. Thank you.