search

SpiderLabs / owasp-modsecurity-crs

OWASP ModSecurity Core Rule Set (CRS) Project (Official Repository)
https://modsecurity.org/crs
Apache License 2.0
2.45k stars 724 forks source link

Cannot get reverse proxy to work #1464

Closed lfarndt closed 4 years ago

lfarndt commented 5 years ago

I just want to use the provided Dockerfiles to create a container doing reverse proxying to another container with my application.

I tried with all the Dockerfiles in https://github.com/SpiderLabs/owasp-modsecurity-crs/tree/v3.2/dev/util/docker

I start my container with docker run -d -p 8765:80 --name security-proxy -e "PARANOIA=5" -e PROXY=1 -e / UPSTREAM=192.168.0.90:666 --rm owasp/modsecurity-crs

What i expect to see is the first page of my app.

I only see the "hello world" from the proxy

dune73 commented 5 years ago

@franbuehler, can you help?

franbuehler commented 5 years ago

We are currently revising our Docker images.

Reverse Proxy mode is currently only available if you rebuild the images from the provided parent modsecurity Dockerfiles available at https://github.com/CRS-support/modsecurity-docker. The Reverse Proxy mode is now implemented in this parent modsecurity image during build time. Default is SETPROXY=False. Also see: https://hub.docker.com/r/owasp/modsecurity

What you can do:

git clone https://github.com/CRS-support/modsecurity-docker.git
cd modsecurity-docker
git checkout v2/apache-apache
docker build --build-arg SETPROXY=True -t my-modsec .
cd ..
git clone https://github.com/SpiderLabs/owasp-modsecurity-crs.git
cd owasp-modsecurity-crs/
cd util/docker/
vim Dockerfile-2.9-apache  <- set your built image from above behind the FROM 
docker build -t my-modsec-crs -f Dockerfile-2.9-apache .

docker run -dti --rm -p 80:80 -e PARANOIA=1 -e PROXYLOCATION=http://192.168.1.100:8000 my-modsec-crs

There are even more CRS variables available now, see: https://github.com/SpiderLabs/owasp-modsecurity-crs/blob/v3.2/dev/util/docker/README.md

Or you use the provided docker-compose.yaml: https://github.com/SpiderLabs/owasp-modsecurity-crs/blob/v3.2/dev/util/docker/docker-compose.yaml

lfarndt commented 5 years ago

Thx, for your fast answer. Now there seems to be another problem, so the container crashes after the start

Traceback (most recent call last): File "<stdin>", line 3, in <module> File "/usr/lib/python2.7/UserDict.py", line 40, in __getitem__ raise KeyError(key) KeyError: 'ANOMALYIN' AH00526: Syntax error on line 15 of /usr/local/apache2/conf/extra/httpd-logging-before-modsec.conf: Unrecognized LogFormat directive %x

franbuehler commented 5 years ago

I see the problem. The problem will go away when you activate TLS during build time of the modsecurity container.

--build-arg SETTLS=True

docker build --build-arg SETPROXY=True --build-arg SETTLS=True -t my-modsec .

We will talk about these containers in a workshop tomorrow. Maybe we need to rethink some details. Sorry for the circumstances!

tlaukkan commented 5 years ago

Hi

Thank you for this valuable work. Would it be possible to provide at least one prebuilt image with proxy enabled?

Kind regards, Tommi

lfarndt commented 5 years ago

@franbuehler Thx for your work. I got it working. You have to put a trailing / after the Proxylocation. Otherwise the apache revers proxy throws an DNS Error docker run -dti --rm -p 80:80 -e PARANOIA=1 -e PROXYLOCATION=http://192.168.1.100:8000/ my-modsec-crs

tlaukkan commented 5 years ago

Following the above directions to build my own images on startup I get the following error:

AH00526: Syntax error on line 15 of /usr/local/apache2/conf/extra/httpd-logging-before-modsec.conf: Unrecognized LogFormat directive %x

lfarndt commented 5 years ago

Did you use the build statement from above with

docker build --build-arg SETPROXY=True --build-arg SETTLS=True -t my-modsec .

I got the same error first. (see above!)

tlaukkan commented 5 years ago

Uh thanks :)

franbuehler commented 5 years ago

Thank you for your feedback @lfarndt and @tlaukkan. We appreciate that very much. I see, we have to improve that.

franbuehler commented 4 years ago

I have to check why the README https://github.com/SpiderLabs/owasp-modsecurity-crs/blob/v3.2/dev/util/docker/README.md is not taken for DockerHubs documentation: https://hub.docker.com/r/owasp/modsecurity-crs.

franbuehler commented 4 years ago

I realised that the DockerHub documentation contains a wrong Dockerfile and README. This is confusing for the users. Do we want to provide a blog post or another Docker documentation on coreruleset.org. There I could describe what the official CRS Docker Images supports and how it can be used? I would not put too much effort into this, because we got Docker image updates or new images promised. But a little explanation would be helpful.

Or any other ideas?

pl853 commented 4 years ago

We are currently revising our Docker images.

Reverse Proxy mode is currently only available if you rebuild the images from the provided parent modsecurity Dockerfiles available at https://github.com/CRS-support/modsecurity-docker. The Reverse Proxy mode is now implemented in this parent modsecurity image during build time. Default is SETPROXY=False. Also see: https://hub.docker.com/r/owasp/modsecurity

What you can do:

git clone https://github.com/CRS-support/modsecurity-docker.git
cd modsecurity-docker
git checkout v2/apache-apache
docker build --build-arg SETPROXY=True -t my-modsec .
cd ..
git clone https://github.com/SpiderLabs/owasp-modsecurity-crs.git
cd owasp-modsecurity-crs/
cd util/docker/
vim Dockerfile-2.9-apache  <- set your built image from above behind the FROM 
docker build -t my-modsec-crs -f Dockerfile-2.9-apache .

docker run -dti --rm -p 80:80 -e PARANOIA=1 -e PROXYLOCATION=http://192.168.1.100:8000 my-modsec-crs

There are even more CRS variables available now, see: https://github.com/SpiderLabs/owasp-modsecurity-crs/blob/v3.2/dev/util/docker/README.md

Or you use the provided docker-compose.yaml: https://github.com/SpiderLabs/owasp-modsecurity-crs/blob/v3.2/dev/util/docker/docker-compose.yaml

dear @franbuehler , First of all thank you, and al the people working at and for SpiderLabs, very much for everything you are doing for the community.

At the moment I'm building a stack containing a Gunicorn Web Server with the Python-Flask Microwebframework and a NGINX Web Server with ModSecurity installed running in docker containers.

The post that i quoted worked for me, but now I'm trying to do it with NGINX v3.0.

Here is what is did:

For v2 Apache (WORKS):

  1. git clone https://github.com/CRS-support/modsecurity-docker.git
  2. cd modsecurity-docker
  3. git checkout v2/apache-apache
  4. docker build --build-arg SETPROXY=True -t my-modsec-apache2 .
  5. cd ..
  6. git clone https://github.com/SpiderLabs/owasp-modsecurity-crs.git
  7. cd owasp-modsecurity-crs/
  8. cd util/docker/
  9. vim Dockerfile-2.9-apache <- set your built image to my-modsec-apache2
  10. docker build -t my-modsec-crs-apache2 -f Dockerfile-2.9-apache .
  11. docker run -ti --rm -p 80:80 -e PARANOIA=1 -e PROXYLOCATION=http://192.168.0.42:8000/ my-modsec-crs-apache2

For v3 Apache: (Doesn't work)

  1. git clone https://github.com/CRS-support/modsecurity-docker.git
  2. cd modsecurity-docker
  3. git checkout v3/apache-apache
  4. docker build --build-arg SETPROXY=True -t my-modsec-apache3 .
  5. cd ..
  6. git clone https://github.com/SpiderLabs/owasp-modsecurity-crs.git
  7. cd owasp-modsecurity-crs/
  8. cd util/docker/
  9. vim Dockerfile-3.0-apache <- set your built image to my-modsec-apache3
  10. docker build -t my-modsec-crs-apache3 -f Dockerfile-3.0-apache.

Then tried: docker run -ti --rm -p 80:80 -e PARANOIA=1 -e PROXYLOCATION=http://192.168.0.42:8000/ my-modsec-crs-apache3

And: docker run -ti --rm -p 80:80 -e PARANOIA=1 -e PROXY=1 -e UPSTREAM=192.168.0.42:8000/ my-modsec-crs-apache3

When navigating to localhost it says :The connection was reset in the browser and in the console it says: AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using 172.17.0.2. Set the 'ServerName' directive globally to suppress this message

For v3 NGINX: (Doesn't work)

  1. git clone https://github.com/CRS-support/modsecurity-docker.git
  2. cd modsecurity-docker
  3. git checkout v3/nginx-nginx
  4. docker build --build-arg SETPROXY=True -t my-modsec-nginx3 .
  5. cd ..
  6. git clone https://github.com/SpiderLabs/owasp-modsecurity-crs.git
  7. cd owasp-modsecurity-crs/
  8. cd util/docker/
  9. vim Dockerfile-3.0-nginx <- set your built image to my-modsec-nginx3
  10. docker build -t my-modsec-crs-nginx3 -f Dockerfile-3.0-nginx .

Then Tried: docker run -ti --rm -p 80:80 -e PARANOIA=1 -e PROXYLOCATION=http://192.168.0.42:8000/ my-modsec-crs-nginx3

And: docker run -ti --rm -p 80:80 -e PARANOIA=1 -e PROXY=1 -e UPSTREAM=192.168.0.42:8000 my-modsec-crs-nginx3

Which (when navigating to localhost) both resulted in the error:

terminate called after throwing an instance of 'std::logic_error'
  what():  basic_string::_M_construct null not valid
2019/10/03 15:37:30 [alert] 1#1: worker process 40 exited on signal 6 (core dumped)

I also tried the above mentioned cases with the original images provided in CRS's provided at https://github.com/SpiderLabs/owasp-modsecurity-crs/blob/v3.2/dev/util/docker/ ( e.g FROM owasp/modsecurity:3.0-nginx) In this case it just goes to the default page. In the case of NGINX, the NGINX welcome screen and with Apache it displays a screen saying IT WORKS. But when doing it this way the reverse proxy is not working.

I also tried to set all the environment variables in the docker files before building them.

I also tried to replace my local IP address with the docker IP that represents the container I want to pass, in my case it is 172.19.0.2 I have the feeling I'm real close to the solution but I was hoping you could help me out a bit.

I also noticed that (in docker-entrypoint.sh), in case the webserver == NGINX, the WEBSERVER_ARGUMENTS is empty : 


if [ $WEBSERVER = "Apache" ]; then
  if [ ! -z $PROXY ]; then
    if [ $PROXY -eq 1 ]; then
      WEBSERVER_ARGUMENTS='-D crs_proxy'
      if [ -z "$UPSTREAM" ]; then
        export UPSTREAM=$(/sbin/ip route | grep ^default | perl -pe 's/^.*?via ([\d.]+).*/$1/g'):81
      fi
    fi
  fi
elif [ $WEBSERVER = "Nginx" ]; then
  WEBSERVER_ARGUMENTS=''
fi

I was wondering if this could be the problem.

Sorry for the long post and thanks a lot in advance.

Yours sincerely, Pieter Lems

pl853 commented 4 years ago

Dear @franbuehler After a lot of trying I managed to get it working without the crs rules. When rebuilding the base image I had to set the proxy location as well: docker build --build-arg SETPROXY=True --build-arg PROXYLOCATION=192.168.0.55:8000 -t my-modsec-nginx3

Then follow the same procedure as mentioned above:

cd ..
git clone https://github.com/SpiderLabs/owasp-modsecurity-crs.git
cd owasp-modsecurity-crs/
cd util/docker/
vim Dockerfile-3.0-nginx <- set your built image to my-modsec-nginx3
docker build -t my-modsec-crs-nginx3 -f Dockerfile-3.0-nginx .

Now I only have to build the owap-modsecurity-crs image again and run in it with the command: docker run -ti 80:80 my-modsec-crs-nginx3

I was hoping there is a way to pass the proxylocation argument in the command mentioned above instead of when building the base image.

In the dockerfile I had to change the line: printf "include /etc/modsecurity.d/owasp-crs/crs-setup.conf\ninclude /etc/modsecurity.d/owasp-crs/rules/*.conf" >> /etc/modsecurity.d/include.conf && \

to printf "include /etc/modsecurity.d/owasp-crs/crs-setup.conf" >> /etc/modsecurity.d/include.conf && \

Because if /etc/modsecurity.d/owasp-crs/rules/*.conf is included it still throws the error:

terminate called after throwing an instance of 'std::logic_error'
  what():  basic_string::_M_construct null not valid

I think the error is related to another problem since there was an issue opened yesterday where this problem is mentioned. (issue 2180 in the ModSecurity repo)

Again thanks in advance! Yours sincerely, Pieter Lems

pl853 commented 4 years ago

Well I fixed it, the only thing I had to do was load the modules in my custom nginx configuration files. facepalm

For anyone that is also struggling with this: At the top of your nginx.conf file add: load_module modules/ngx_http_modsecurity_module.so;

Then for the upstream server add :

http {
    modsecurity on;
    modsecurity_rules_file /etc/modsecurity.d/include.conf;

    upstream app {
        server app:8000;
    }
    server {
        listen       8000;
        server_name  app.localhost;

        location / {
            proxy_pass http://app;
        } 
}
mbuchner commented 4 years ago

Could somebody provide an instruction how to build those images with proxy and / or tls ... I followed the instructions above -> here are my exact steps ....

and afterwards:

So far so good - build shows no errors ... -> Then run the build:

Container stops after 3 seconds -> log states:

docker logs

standard_init_linux.go:211: exec user process caused "no such file or directory"`

I also tried it with "https://github.com/CRS-support/modsecurity-docker.git" -> "git checkout v3/apache-apache" and then with "Dockerfile-3.0-apache" -> also no luck ... grrrrr

franbuehler commented 4 years ago

Sorry for the inconvenience, @mbuchner!

You use the standard v2/apache-apache image. Because this is the default image we support TLS and PROXY support as a default in this image. You do not need to build this image.

But of course you can build the image from source.

Your commands look good and it "works on my machine". But we can shorten some of the commands.

The minimum commands you use to rebuild the v2/apache-apache ModSecurity image are:

You don't need to set SETTLS and SETPROXY, they have a default of 1 for v2/apache-apache. But of course you can set them. You don't need to set PROXYLOCATION, you can set it later in the docker run command, it's an ENV, not an ARG.

The commands to rebuild the CRS image (Dockerfile-2.9-apache) are correct.

To run the image, only PROXYLOCATION is used. PROXY and UPSTREAM are not supported/needed anymore: docker run -p 8881:80 -p 8443:443 -dti -e PARANOIA=1 -e PROXYLOCATION=http://172.52.10.31:9180/ cat-modsec-crs-v1.0.1

As I said I am sorry for the inconvenience. We are still working on these images and hope to fix this problems once and for all very soon. In the meantime please reach out to us to get support.

github-actions[bot] commented 4 years ago

This issue has been open 120 days with no activity. Remove the stale label or comment, or this will be closed in 14 days