Fail2Ban Banning but not Blocking

[abcde57219]

Thank you for providing the information on this repository. I set up Fail2Ban shortly after installing BitwardenRS a month or two ago. At that time, Fail2Ban seemed to be working. I checked it this weekend, and although my IP is being banned in the logs, I can still successfully log into BitwardenRS. I have also setup notifications and receive an e-mail that I have been banned when I try to log in 5 times.

I am running BitwardenRS using the bitwardenrs/server:latest docker image and Fail2Ban using crazymax/fail2ban:latest docker image. I am using a Synology NAS with a reverse proxy to access BitwardenRS. A couple of approaches I have tried to fix the issue include:

• Adding the NAS ports and the reverse proxy port in addition to the BitwardenRS container's ports.
• Adding /action.d/iptables-allports.local to try to kill any existing connections as I read this could be an issue. I only receive error messages in the log when I try this.
• Adding chain=FORWARD to the /jail.d/bitwarden.local file. However, I did not think this necessary because I am using a reverse proxy.
• I have made sure the timezone is the same in all areas.
• I tried on my cellphone using the cellphone network IP.

Please let me know if you think there is anything else I can try.

Here is from my logs:

2020-04-19 01:17:59,883 fail2ban.filter   [1]: INFO    [bitwarden] Found myip - 2020-04-19 01:17:59
2020-04-19 01:18:01,059 fail2ban.filter   [1]: INFO    [bitwarden] Found myip - 2020-04-19 01:18:01
2020-04-19 01:18:01,913 fail2ban.filter   [1]: INFO    [bitwarden] Found myip - 2020-04-19 01:18:01
2020-04-19 01:18:02,694 fail2ban.filter   [1]: INFO    [bitwarden] Found myip - 2020-04-19 01:18:02
2020-04-19 01:18:03,488 fail2ban.filter   [1]: INFO    [bitwarden] Found myip - 2020-04-19 01:18:03
2020-04-19 01:18:03,714 fail2ban.actions  [1]: NOTICE  [bitwarden] Ban myip
2020-04-19 01:18:04,326 fail2ban.filter   [1]: INFO    [bitwarden] Found myip - 2020-04-19 01:18:04
2020-04-19 01:18:05,230 fail2ban.filter   [1]: INFO    [bitwarden] Found myip - 2020-04-19 01:18:05
2020-04-19 01:18:06,091 fail2ban.filter   [1]: INFO    [bitwarden] Found myip - 2020-04-19 01:18:06
2020-04-19 01:18:07,562 fail2ban.filter   [1]: INFO    [bitwarden] Found myip - 2020-04-19 01:18:07
2020-04-19 01:18:09,345 fail2ban.filter   [1]: INFO    [bitwarden] Found myip - 2020-04-19 01:18:09
2020-04-19 01:18:10,933 fail2ban.actions [1]: NOTICE  [bitwarden] myip already banned

Here is from my /jail.d/bitwarden.local:

[bitwarden]
enabled = true
port = 80,3012
filter = bitwarden
action = iptables-allports[name=Bitwarden]
         sendmail-whois[name=Bitwarden, dest=myemail@email.com]
logpath = /bitwarden/bitwarden.log
maxretry = 5
bantime = 14400
findtime = 14400

Here is from my /filter.d/bitwarden.local:

[INCLUDES]
before = common.conf

[Definition]
failregex = ^.*Username or password is incorrect\. Try again\. IP: <ADDR>\. Username:.*$
ignoreregex =

Here is from my /action.d/iptables-common.local:

[Init]
blocktype = DROP
[Init?family=inet6]
blocktype = DROP

Here is from my docker-compose.yml:

version: '3'
services:
  fail2ban:
    container_name: fail2ban
    restart: always
    image: crazymax/fail2ban:latest
    environment: 
    - TZ=Etc/UTC
    - F2B_DB_PURGE_AGE=1d
    - F2B_LOG_TARGET=/data/fail2ban.log
    - F2B_LOG_LEVEL=INFO
    - F2B_IPTABLES_CHAIN=INPUT
    - SSMTP_HOST=smtp.email.com
    - SSMTP_PORT=port
    - SSMTP_HOSTNAME=email
    - SSMTP_USER=myemail@email.com
    - SSMTP_PASSWORD=mypassword
    - SSMTP_TLS=YES
    - SSMTP_STARTTLS=Yes

    volumes:
    - /volume1/docker/fail2ban:/data
    - /volume1/docker/bitwarden:/bitwarden:ro

    network_mode: "bridge"

    privileged: true
    cap_add:
        - NET_ADMIN
        - NET_RAW

[ZenoBell]

I'm getting exactly the same issue with the difference that i'm using fail2ban directly installed in an ubuntu server. I can see clearly from the logs in /var/log/fail2ban.log that the ip is getting banned but if i try to login with the my correct credentials bitwarden allows me. I'm using traefik 1 & docker. Any suggestion ?

[philippjess]

I guess your reverse proxy is also a docker container, then you are adding to the wrong chain, in the bw jail use DOCKER-USER chain like this: action = iptables-allports[name=bitwarden, chain=DOCKER-USER]

Works for me with a nginx reverse proxy and the fail2ban container. Also use the ports of your proxy.

[ZenoBell]

hmm thanx for the suggestion but i have no idea why it's not working : After some attempts i see that the loading button is just looping giving me hope that the ip is banned but then refreshing the page and entering my normal credentials is giving me access. Also I'm making the test with a phone with cellular data and i see that the ip doesn't matches the ones in the log probably because of docker networking but still not sure if it's ok. What i tried :

1. action = iptables-allports[name=bitwarden]
2. action = iptables-allports[name=bitwarden, chain=DOCKER-USER]
3. action = iptables-allports[name=bitwarden, chain=FORWARD]

i'm always restarted the container but nothing of these seems to work.

[philippjess]

Maybe in your reverse proxy config you forgot to forward the real IP, so f2b is banning docker ips.

[ZenoBell]

The F2B_IPTABLES_CHAIN env var was removed according to the latest Readme of crazy-max. hmm so with docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' container_name_or_id i saw what's the ip in docker and it's not the same with the one that fail2bann blocks. Checking a little better i found that is a cloudflare one as the connection is made through cloudflare cdn. There is an option in Cloudflare to enable an option that adds to the request a True-Client-IP header but it's unfortunately only for enterprise plans. In Caddy there is also a plugin for a true ip but i didnt found anything official in Traefik. What i found although is the configuration of this guy https://gist.github.com/acundari/9bdcf2ba0c0f8a4bf59a21d06da35612 so if i understood correct if i specify a folder to output the traefik logs in the traefik yaml as the default logs are written in stdout with the type of logs specified in the toml (info,debugging,etc) and create the above jails and filters it should work as it blocks IPs that have HTTP Basic Authentication failures.

edit : i found also this that could solve the issue, an example of traefik configuration directly from crazy-max https://github.com/crazy-max/docker-fail2ban/tree/master/examples/jails/traefik

[alexanderharm]

I'm having exactly the same problem. Unfortunately logging seems not to be possible to troubleshoot. Fail2Ban makes the entry in iptables but I can still connect from the banned IP with different devices. No idea what is going on here...

[N1c093]

I have exactly the same problem. I also found the cause in my situation, but no solution.

I run an nginx reverse proxy on an raspberry pi which forwards the traffic to my synology nas, where bitwarden and fail2ban is running. If fail2ban is banning the external ip address there is no blocking happening, because all the traffic comes from the reverse proxy (internal ip address)

There seems to be an workaround, where you can block traffic based on its header. Sadly this doesn't work on synology: https://centos.tips/fail2ban-behind-a-proxyload-balancer/

[BlackDex]

Firewalls (like iptables) can only block the IP which is sending the actual request to where fail2ban is running. So if you have a construction like this:

[router] --> [reverse-proxy] --> [bitwarden_rs-host] what will happen is that the [bitwarden_rs-host] only sees the IP from the [reverse-proxy] on a network level.

The reverse proxy can send the actual IP using a header, but that is about it using simple configurations. There are some options to use a transparent reverse proxy, which will pass the actual remote IP to the [bitwarden_rs-host].

• https://www.nginx.com/blog/ip-transparency-direct-server-return-nginx-plus-transparent-proxy/
• https://www.haproxy.com/blog/howto-transparent-proxying-and-binding-with-haproxy-and-aloha-load-balancer/

I also have nginx in-front of my bitwarden_rs, and i use the following filter.d file for my bitwarden_rs blocking. This happens on my nginx reverse proxy node, which does receive the correct IP in the TCP Stack. /etc/fail2ban/filter.d/nginx-bitwardenrs.local

# fail2ban filter configuration for bitwarden_rs reverse nginx proxy
[INCLUDES]
before = common.conf

[Definition]
failregex = ^<HOST> \- \S+ \[\] \"(GET|POST|CONNECT|PUT|DELETE) \/.*? \S+\" 400 .+$
ignoreregex =
datepattern = {^LN-BEG}%%ExY(?P<_sep>[-/.])%%m(?P=_sep)%%d[T ]%%H:%%M:%%S(?:[.,]%%f)?(?:\s*%%z)?
              ^[^\[]*\[({DATE})
              {^LN-BEG}

And then in /etc/fail2ban/jail.local i have:

[nginx-bitwardenrs]
enabled  = true
port     = http,https
# logpath  = %(nginx_access_log)s
logpath = /var/log/nginx/bitwardenrs-access.log
maxretry = 4

A failed log entry looks like this:

333.111.222.444 - - [07/Oct/2020:15:37:33 +0000] "POST /identity/connect/token HTTP/2.0" 400 227 "https://bitwarden.example.tld/" "Mozilla/5.0 (X11; Linux x86_64; rv:82.0) Gecko/20100101 Firefox/82.0"

[wcjxixi]

@abcde57219

I'm having exactly the same problem. maybe the timezone of the docker container dosn't match the timezone of the host.

verify that the timezone of the docker container matches the timezone of the host. Check this by comparing the time shown in the logfile with the host OS time. If they differ, there are various ways to fix this. One option is to start docker with the option -e "TZ=<timezone>".

First make sure the timezone of the docker container matches the timezone of the host, additional, Fail2Ban DROP instead REJECT, see https://gist.github.com/antoniocampos/1b8bc607d7b2d4a42e2a6e7df00645d0

[BlackDex]

On the docker host you could run this to verify

echo "Host     : $(date)" ; echo "Container: $(docker exec -it bitwarden date)"

But still that doesn't solve the issue with IP's not being transparently forwarded from the nginx/haproxy reverse proxy to the docker container not receiving the correct IP. Unless the system where docker is installed also is the system where the reverse proxy is located.

[wcjxixi]

@BlackDex

additional, Fail2Ban DROP instead REJECT, see https://gist.github.com/antoniocampos/1b8bc607d7b2d4a42e2a6e7df00645d0

[BlackDex]

That doesn't matter. It will either DROP the packet and thus it will not respond to the request, or it will REJECT using some type of REJECT like icmp-host-unreachable or something. Both will not allow the traffic to go any further. So that will not solve there problem.

[BlackDex]

Closing this ticket because of inactivity. Feel free to continue this discussion on the forum: https://bitwardenrs.discourse.group/