Modsecurity How to get real IP address??

[Hello-Linux]

Our nginx server has gone through a layer of proxy, and the real user address cannot be obtained using REMOTE_ADDR,But our proxy server has a variable ($http_x_forwarded_for) to get the user's real IP address,So what is the solution for modsecurity?

SecRule REMOTE_ADDR "@geoLookup" "chain,id:22,deny,phase:1,log,msg:'Non-China IP address'"
SecRule GEO:COUNTRY_CODE "!@rx CN|HK|TW|MO"

[airween]

You can use the REQUEST_HEADERS:X-Forwarded-For variable instead of REMOTE_ADDR.

[Hello-Linux]

@airween Thank you very much, I will try it

[Dark-Schnitzel]

This is also working for me, but if i you are using apache with mod_remoteip you should turn these off, otherwise this is empty. you can log this also via apache and logformat: LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" https://www.techstacks.com/howto/log-client-ip-and-xforwardedfor-ip-in-apache.html

[Hello-Linux]

@Dark-Schnitzel I use Nginx

[Hello-Linux]

@airween @Dark-Schnitzel This seems to have no effect, my nginx logs show the realip(x_forwarded)

{ "@timestamp": "2020-11-12T14:26:06+08:00", "remote_addr": "100.97.166.33","request_time": "0.000","upstream_response_time": "","status": 200,"x_forwarded": "36.110.73.230","http_referer": "","request": "GET /status HTTP/1.1","upstream_addr": "","body_bytes_sent":637,"request_body":,"http_user_agent": "Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:82.0) Gecko/20100101 Firefox/82.0" }

the nginx.conf logformat is

log_format main escape=json '{ "@timestamp": "$time_iso8601", '
                       '"remote_addr": "$remote_addr",'
                       '"request_time": "$request_time",'
                       '"upstream_response_time": "$upstream_response_time",'
                       '"status": $status,'
                       '"x_forwarded": "$http_x_forwarded_for",'
                       '"http_referer": "$http_referer",'
                       '"request": "$request",'
                       '"upstream_addr": "$upstream_addr",'
                       '"body_bytes_sent":$body_bytes_sent,'
                       '"request_body":$request_body,'
                       '"http_user_agent": "$http_user_agent" }';

and my rules/REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf is

SecRule REQUEST_HEADERS:X-Forwarded-For "@geoLookup" \
    "id:22,\
    deny,\
    phase:1,\
    log,\
    msg:'Non-China IP address',\
    chain"
    SecRule GEO:COUNTRY_CODE "!@rx US" "log"

After adding REQUEST_HEADERS:X-Forwarded-For, my access to the website was not denied. Why?

[airween]

my access to the website was not denied. Why?

What's your goal?

I tried with your examples (X-Forwarded-For is 36.110.73.230 and your SecRule), and I got this in debug.log:

[1605174515] [/] [4] (Rule: 22) Executing operator "GeoLookup against REQUEST_HEADERS:X-Forwarded-For.
[1605174515] [/] [9] Target value: "36.110.73.230" (Variable: REQUEST_HEADERS:X-Forwarded-For)
[1605174515] [/] [9] Matched vars updated.
[1605174515] [/] [9] Saving msg: Non-China IP address
[1605174515] [/] [4] Rule returned 1.
[1605174515] [/] [4] Executing chained rule.
[1605174515] [/] [4] (Rule: 0) Executing operator "Rx" with param "US" against GEO:COUNTRY_CODE.
[1605174515] [/] [9] Target value: "CN" (Variable: GEO:COUNTRY_CODE)
[1605174515] [/] [9] Matched vars updated.
[1605174515] [/] [4] Rule returned 1.

in the error.log:

ModSecurity: Access denied with code 403 (phase 1). Matched "Operator `Rx' with parameter `US' against variable `GEO:COUNTRY_CODE' (Value: `CN' ) 

So, I think the IP address above is from CN, but you deny the request if the COUNTRY_CODE is not US. If you want to allow the request only from CN, you should replace the US by CN.

[Hello-Linux]

My problem now is that the configuration does not take effect. My ip address is China and only the United States is allowed in the configuration. However, my website from China is not denied and can be accessed normally, so why does my configuration not take effect?

[Hello-Linux]

@airween I should be denied access to this website from China, but in reality, I can access it normally. Why is that?

[airween]

What is your SecRuleEngine setting? Could you turn on the debug log, and see what happens?

[Hello-Linux]

@airween Okay, I'll go home and open the debug mode to observe

[Hello-Linux]

@airween

My SecRuleEngine is set to On,and I see the debug log is show below

and my crs-setup.conf is

the rules/REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf is

[airween]

If you only see these lines in your debug.log:

[1605187401] [/test/zyib/h5/youbaohui/favicon.ico] [4] (Rule: 12345) Executing operator "GeoLookup against REQUEST_HEADERS:X-Forwarded-For.
[1605187401] [/test/zyib/h5/youbaohui/favicon.ico] [4] Rule returned 0.

then it means that header isn't presented.

You must see these lines too:

[1605187401] [/test/zyib/h5/youbaohui/favicon.ico] [4] (Rule: 12345) Executing operator "GeoLookup against REQUEST_HEADERS:X-Forwarded-For.
[1605187401] [/test/zyib/h5/youbaohui/favicon.ico] [9] Target value: "36.110.73.230" (Variable: REQUEST_HEADERS:X-Forwarded-For)
[1605187401] [/test/zyib/h5/youbaohui/favicon.ico] [9] Matched vars updated.
[1605187401] [/test/zyib/h5/youbaohui/favicon.ico] [9] Saving msg: Non-China IP address
[1605187401] [/test/zyib/h5/youbaohui/favicon.ico] [4] Rule returned 1.

from 2nd to 4th.

Btw, what is the value of the SecDebugLogLevel in your conf? Set it to 9 and check again.

[Hello-Linux]

@airween
I have set SecDebugLogLevel to 9,the logs is below,and there is no "Matched vars updated" phrase in my debug.log Is it because the ip address does not match? I changed several public IP addresses and it seems that it doesn’t work.

and the maxmind website show my ip is CN,why modsecurity can't match it?

[Hello-Linux]

The I change the rule is

SecRule REQUEST_HEADERS:X-Forwarded-For "@geoLookup" \
    "id:12345,\
    deny,\
    phase:1,\
    log,\
    msg:'Non-China IP address',\
    chain"
    SecRule GEO:COUNTRY_CODE "!@rx CN" "nolog"

I can also visit the website normally using a foreign IP address,not deny!

[Hello-Linux]

@airween Sometimes the log show Database is not open. Use: SecGeoLookupDb directive,but the SecGeoLookupDB I have set

[Hello-Linux]

@airween I use wireshark to grab the data packet of the proxy server, and you can also find the header in the request header-X-Forwarded-For: 52.29.177.64,so I think the modsecurity can get the header-X-Forwarded-For?

GET /dev/website2018/bImsFront/manage/index.html HTTP/1.1
RemoteIp: 52.29.177.64
Host: ceshiweb.xxxxx.com
X-Forwarded-For: 52.29.177.64
Connection: close
pragma: no-cache
cache-control: no-cache
upgrade-insecure-requests: 1
user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/61.0.3163.100 Chrome/61.0.3163.100 Safari/537.36 PingdomPageSpeed/1.0 (pingbot/2.0; +http://www.pingdom.com/)
accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
accept-encoding: gzip, deflate, br
accept-language: en-US,en;q=0.8
X-Forwarded-Proto: https

[airween]

@Hello-Linux,

could you show me the final part of the output of the ./configure ... script?

I mean:

configure: creating ./config.status

ModSecurity - v3.0.4-92-gd72be1c4 for Linux

 Mandatory dependencies
...
...

And what did you get for this command?

ls -l /usr/local/openresty/nginx/conf/modsecurity/geoip/geoLite2-Country.mdb

Please don't insert screenshot, just copy the text and insert here. Use the "Insert code" item at the menu bar.

[Hello-Linux]

@airween the status is

[root@devtestjenkins ModSecurity]# ./config.status
config.status: creating modsecurity.pc
config.status: creating Makefile
config.status: creating doc/Makefile
config.status: creating src/Makefile
config.status: creating others/Makefile
config.status: creating tools/Makefile
config.status: creating tools/rules-check/Makefile
config.status: creating test/Makefile
config.status: creating test/benchmark/Makefile
config.status: creating examples/Makefile
config.status: creating examples/simple_example_using_c/Makefile
config.status: creating examples/multiprocess_c/Makefile
config.status: creating examples/reading_logs_with_offset/Makefile
config.status: creating examples/reading_logs_via_rule_message/Makefile
config.status: creating examples/using_bodies_in_chunks/Makefile
config.status: creating src/config.h
config.status: src/config.h is unchanged
config.status: executing depfiles commands
config.status: executing libtool commands

and the geoLite2-Country.mdb is GeoLite2-Country.mmdb

[root@devtestjenkins modsecurity]# ls -alh /usr/local/openresty/nginx/conf/modsecurity/geoip/GeoLite2-Country.mmdb
-rw-r--r-- 1 root root 3.8M Nov 11 10:06 /usr/local/openresty/nginx/conf/modsecurity/geoip/GeoLite2-Country.mmdb

[airween]

Please run the configure script as you use it (with your options), and show me the last part of the output, from the line configure: creating...

[Hello-Linux]

@airween the out put is

[root@devtestjenkins ModSecurity]# ./configure
checking for a BSD-compatible install... /bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking for g++... g++
checking whether the C++ compiler works... yes
checking for C++ compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking for style of include used by make... GNU
checking dependency style of g++... gcc3
checking for gcc... gcc
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking dependency style of gcc... gcc3
checking for ar... ar
checking the archiver (ar) interface... ar
checking whether make sets $(MAKE)... (cached) yes
checking for pkg-config... /bin/pkg-config
checking pkg-config is at least version 0.9.0... yes
configure: Nothing about YAJL was informed during the configure phase. Trying to detect it on the platform...
configure: using YAJL v2.0.4
configure: Nothing about GeoIP was informed during the configure phase. Trying to detect it on the platform...
configure: using GeoIP v1.5.0
configure: Nothing about MaxMind was informed during the configure phase. Trying to detect it on the platform...
configure: MaxMind library was not found
configure: Nothing about LMDB was informed during the configure phase. Trying to detect it on the platform...
configure: LMDB is disabled by default.
*** LOOKING AT PATH:  /usr/lib
*** LOOKING AT PATH:  /usr/local/lib
*** LOOKING AT PATH:  /usr/local/fuzzy
*** LOOKING AT PATH:  /usr/local/libfuzzy
*** LOOKING AT PATH:  /usr/local
*** LOOKING AT PATH:  /opt
*** LOOKING AT PATH:  /usr
configure: SSDEEP headers found at: /usr/include
*** LOOKING AT PATH:  /usr/lib64
configure: SSDEEP library found at: /usr/lib64//libfuzzy.so
configure: SSDEEP headers found at: /usr/include
configure: using SSDEEP v
*** LOOKING AT PATH:  /usr/lib
*** LOOKING AT PATH:  /usr/local/lib
*** LOOKING AT PATH:  /usr/local/lib64
*** LOOKING AT PATH:  /usr/local/lua
*** LOOKING AT PATH:  /usr/local/liblua
*** LOOKING AT PATH:  /usr/local
*** LOOKING AT PATH:  /opt
*** LOOKING AT PATH:  /usr
configure: LUA headers found at: /usr/include
*** LOOKING AT PATH:  /usr/lib64
configure: LUA library found at: /usr/lib64//liblua-5.1.so
configure: LUA headers found at: /usr/include
configure: LUA version from includes: 501
configure: using LUA -llua-5.1
checking for libcurl config script... /usr/bin/curl-config
configure: curl VERSION: 7.29.0
configure: curl LDADD:
checking if libcurl is at least v... yes, 7.29.0
checking if libcurl is linked with gnutls... no
configure: using curl v7.29.0
checking for libxml2 config script... /usr/bin/xml2-config
configure: xml VERSION: 2.9.1
configure: xml CFLAGS: -I/usr/include/libxml2 -DWITH_LIBXML2
configure: xml LDADD: -lxml2 -lz -lm -ldl
checking if libxml2 is at least v2.6.29... yes, 2.9.1
configure: using libxml2 v2.9.1
checking for libpcre config script... /usr/bin/pcre-config
configure: pcre VERSION: 8.32
configure: pcre LDADD: -lpcre
configure: pcre PCRE_LD_PATH: /-lpcre
checking for PCRE JIT... yes
configure: using pcre v8.32
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking string usability... no
checking string presence... no
checking for string... no
checking iostream usability... no
checking iostream presence... no
checking for iostream... no
checking sys/utsname.h usability... yes
checking sys/utsname.h presence... yes
checking for sys/utsname.h... yes
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking how to print strings... printf
checking for a sed that does not truncate output... /bin/sed
checking for fgrep... /bin/grep -F
checking for ld used by gcc... /bin/ld
checking if the linker (/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /bin/nm -B
checking the name lister (/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 1572864
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
checking how to convert x86_64-unknown-linux-gnu file names to x86_64-unknown-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-unknown-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for archiver @FILE support... @
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /bin/nm -B output from gcc object... ok
checking for sysroot... no
checking for mt... no
checking if : is a manifest tool... no
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fPIC -DPIC
checking if gcc PIC flag -fPIC -DPIC works... yes
checking if gcc static flag -static works... no
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/bin/ld -m elf_x86_64) supports shared libraries... yes
checking whether -lc should be explicitly linked in... no
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking for shl_load... no
checking for shl_load in -ldld... no
checking for dlopen... no
checking for dlopen in -ldl... yes
checking whether a program can dlopen itself... yes
checking whether a statically linked program can dlopen itself... yes
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... yes
checking how to run the C++ preprocessor... g++ -E
checking for ld used by g++... /bin/ld -m elf_x86_64
checking if the linker (/bin/ld -m elf_x86_64) is GNU ld... yes
checking whether the g++ linker (/bin/ld -m elf_x86_64) supports shared libraries... yes
checking for g++ option to produce PIC... -fPIC -DPIC
checking if g++ PIC flag -fPIC -DPIC works... yes
checking if g++ static flag -static works... no
checking if g++ supports -c -o file.o... yes
checking if g++ supports -c -o file.o... (cached) yes
checking whether the g++ linker (/bin/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... (cached) GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
Checking platform... Identified as Linux
checking for doxygen... /bin/doxygen
checking for perl... /bin/perl
checking for valgrind... no
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating modsecurity.pc
config.status: creating Makefile
config.status: creating doc/Makefile
config.status: creating src/Makefile
config.status: creating others/Makefile
config.status: creating tools/Makefile
config.status: creating tools/rules-check/Makefile
config.status: creating test/Makefile
config.status: creating test/benchmark/Makefile
config.status: creating examples/Makefile
config.status: creating examples/simple_example_using_c/Makefile
config.status: creating examples/multiprocess_c/Makefile
config.status: creating examples/reading_logs_with_offset/Makefile
config.status: creating examples/reading_logs_via_rule_message/Makefile
config.status: creating examples/using_bodies_in_chunks/Makefile
config.status: creating src/config.h
config.status: src/config.h is unchanged
config.status: executing depfiles commands
config.status: executing libtool commands

ModSecurity - v3.0.4-92-gd72be1c for Linux

 Mandatory dependencies
   + libInjection                                  ....v3.9.2-30-gbf234eb
   + SecLang tests                                 ....d03f4c1

 Optional dependencies
   + GeoIP/MaxMind                                 ....found
      * (GeoIP) v1.5.0
         -lGeoIP  , -I/usr/include/
   + LibCURL                                       ....found v7.29.0
      -lcurl  ,  -DWITH_CURL
   + YAJL                                          ....found v2.0.4
      -lyajl  , -DWITH_YAJL
   + LMDB                                          ....disabled
   + LibXML2                                       ....found v2.9.1
      -lxml2 -lz -lm -ldl, -I/usr/include/libxml2 -DWITH_LIBXML2
   + SSDEEP                                        ....found
      -lfuzzy -L/usr/lib64/, -DWITH_SSDEEP -I/usr/include
   + LUA                                           ....found v501
      -llua-5.1 -L/usr/lib64/, -DWITH_LUA -DWITH_LUA_5_1 -I/usr/include

 Other Options
   + Test Utilities                                ....enabled
   + SecDebugLog                                   ....enabled
   + afl fuzzer                                    ....disabled
   + library examples                              ....enabled
   + Building parser                               ....disabled
   + Treating pm operations as critical section    ....disabled

[airween]

MaxMind support seems to be missing:

configure: Nothing about MaxMind was informed during the configure phase. Trying to detect it on the platform...
configure: MaxMind library was not found

  + GeoIP/MaxMind                                 ....found
      * (GeoIP) v1.5.0
         -lGeoIP  , -I/usr/include/

Try to install the libmaxmind library and recompile the engine.

[Hello-Linux]

@airween ok, I will install libmaxminddb and libmaxminddb-devel

[Hello-Linux]

@airween When I install libmaxminddb and libmaxminddb-devel

libmaxminddb.x86_64                                                        1.2.0-6.el7                                                   @base
libmaxminddb-devel.x86_64                                                  1.2.0-6.el7                                                   @base

the ./configure show this

ModSecurity - v3.0.4-92-gd72be1c for Linux

 Mandatory dependencies
   + libInjection                                  ....v3.9.2-30-gbf234eb
   + SecLang tests                                 ....d03f4c1

 Optional dependencies
   + GeoIP/MaxMind                                 ....found
      * (MaxMind) v
         /usr/lib64//libmaxminddb.so, /usr/include, -DWITH_MAXMIND -I/usr/include
      * (GeoIP) v1.5.0
         -lGeoIP  , -I/usr/include/
   + LibCURL                                       ....found v7.29.0
      -lcurl  ,  -DWITH_CURL
   + YAJL                                          ....found v2.0.4
      -lyajl  , -DWITH_YAJL
   + LMDB                                          ....disabled
   + LibXML2                                       ....found v2.9.1
      -lxml2 -lz -lm -ldl, -I/usr/include/libxml2 -DWITH_LIBXML2
   + SSDEEP                                        ....found
      -lfuzzy -L/usr/lib64/, -DWITH_SSDEEP -I/usr/include
   + LUA                                           ....found v501
      -llua-5.1 -L/usr/lib64/, -DWITH_LUA -DWITH_LUA_5_1 -I/usr/include

but when I run make is show some error blow

utils/geo_lookup.cc:124:78: error: invalid conversion from 'const MMDB_s*' to 'MMDB_s*' [-fpermissive]
         r = MMDB_lookup_string(&mmdb, target.c_str(), &gai_error, &mmdb_error);
                                                                              ^
In file included from ../src/utils/geo_lookup.h:22:0,
                 from utils/geo_lookup.cc:25:
/usr/include/maxminddb.h:195:33: error:   initializing argument 1 of 'MMDB_lookup_result_s MMDB_lookup_string(MMDB_s*, const char*, int*, int*)' [-fpermissive]
     extern MMDB_lookup_result_s MMDB_lookup_string(MMDB_s *const mmdb,

[airween]

Your libmaxminddb is too old.

See issue #2254 (and the solution too).

[Hello-Linux]

@airween Ha Ha....Finally succeeded, thank you very much!

[dangdh16]

Hi, anyone still here. I got some error. I had installed libmaxminddb and libmaxminddb-devel successed. But still same log.