Failed to load locate the unicode map file from: unicode.mapping 20127

[tinyoafman]

Describe the bug

v3 of ModSecurity - pulling from master branch.

Running mod security on Nginx inside docker container using ModSecurity nginx connector and getting the following error when the container starts up:

"modsecurity_rules_file" directive Rules error. File: /etc/nginx/modsec/modsecurity.conf. Line: 192. Column: 17. Failed to load locate the unicode map file from: unicode.mapping 20127 Looking at: 'unicode.mapping 20127', 'unicode.mapping 20127', '/etc/nginx/modsec/unicode.mapping 20127', '/etc/nginx/modsec/unicode.mapping 20127'. in /etc/nginx/nginx.conf:17

nginx.conf (/etc/nginx/nginx.conf)

load_module modules/ngx_http_modsecurity_module.so;
error_log /dev/stdout ${LOG_LEVEL};

events {
  worker_connections 1024;
}

http {
    access_log /dev/stdout;
    server_tokens off;
    server {
        listen 443 ssl;
        server_name waf;
        ssl_certificate certs/cert.pem;
        ssl_certificate_key certs/key.pem;
        modsecurity on;
        modsecurity_rules_file /etc/nginx/modsec/main.conf;

        location / {
            proxy_pass https://${LOCATION_ADDRESS};
            modsecurity on;
        }

        error_page 403 404 405 500 501 502 503 504 /error/error.html;

        location ^~ /error/ {
            internal;
            root /usr/share/nginx;
            modsecurity off;
        }

        location = /amihealthy {
            return 200;
        }
    }
}

Dockerfile

FROM nginx:1.15.3-alpine
RUN set -xe && \
    apk --no-cache update && \
        apk add --no-cache --virtual .build-deps \
        gcc \
        libc-dev \
        make \
        openssl-dev \
        pcre-dev \
        zlib-dev \
        linux-headers \
        curl \
        gnupg \
        libxslt-dev \
        gd-dev \
        perl-dev \
        py-pip \
        py-setuptools \
    && apk add --no-cache --virtual .libmodsecurity-deps \
        pcre-dev \
        libxml2-dev \
        automake \
        autoconf \
        g++ \
        flex \
        bison \
        yajl-dev \
    # dependencies that should not be removed
    && apk add --no-cache \
        libtool \
        doxygen \
        geoip \
        geoip-dev \
        yajl \
        libstdc++ \
        git \
        sed \
        python

# install aws cli
RUN set -xe && \
    pip --no-cache-dir install --upgrade pip && \
    pip --no-cache-dir install awscli

# install modsecurity
WORKDIR /opt/ModSecurity
RUN set -xe && \
    git clone -b v3/master --single-branch https://github.com/SpiderLabs/ModSecurity . && \
    git submodule init && \
    git submodule update && \
    ./build.sh && \
    ./configure && make && make install

# install nginx connector
WORKDIR /opt
RUN git clone --depth 1 https://github.com/SpiderLabs/ModSecurity-nginx.git && \
    wget http://nginx.org/download/nginx-$NGINX_VERSION.tar.gz && \
    tar zxvf nginx-$NGINX_VERSION.tar.gz

WORKDIR /opt/nginx-$NGINX_VERSION
RUN set -xe && \
    ./configure --with-compat --add-dynamic-module=../ModSecurity-nginx && \
    make modules && \
    cp objs/ngx_http_modsecurity_module.so /etc/nginx/modules

# install owasp ruleset
WORKDIR /opt
RUN set -xe && \
    git clone -b v3.0/master https://github.com/SpiderLabs/owasp-modsecurity-crs && \
    mv owasp-modsecurity-crs/ /usr/local/

RUN set -xe && \
    mkdir /etc/nginx/modsec && \
    rm -fr /etc/nginx/nginx.conf

COPY conf/nginx/nginx.conf /etc/nginx/nginx.conf.template
COPY conf/modsec/ /etc/nginx/modsec/
COPY conf/owasp/ /usr/local/owasp-modsecurity-crs/
COPY error/ /usr/share/nginx/error/
COPY certs/ /etc/nginx/certs/

# remove unnecessary stuff
RUN set -xe && \
    apk del .build-deps && \
    apk del .libmodsecurity-deps && \
    rm -fr ModSecurity && \
    rm -fr ModSecurity-nginx && \
    rm -fr nginx-$NGINX_VERSION.tar.gz && \
    rm -fr nginx-$NGINX_VERSION && \
    rm -fr /etc/nginx/conf.d

# execute stuff
COPY ./setupEnvAndStart.sh /etc/nginx/
RUN chmod +x /etc/nginx/setupEnvAndStart.sh
CMD ["sh", "-c", ". /etc/nginx/setupEnvAndStart.sh"]

setupEnvAndStart.sh

#! /bin/bash
config=<config>

for i in $(echo $config | sed "s/,/ /g")
do
  export $i
done

envsubst < /etc/nginx/nginx.conf.template > /etc/nginx/nginx.conf
exec nginx -g 'daemon off;'

modsecurity.conf

Used the modsecurity.conf-recommended

Logs and dumps

[emerg] 1#1: "modsecurity_rules_file" directive Rules error. File: /etc/nginx/modsec/modsecurity.conf. Line: 192. Column: 17. Failed to load locate the unicode map file from: unicode.mapping 20127 Looking at: 'unicode.mapping 20127', 'unicode.mapping 20127', '/etc/nginx/modsec/unicode.mapping 20127', '/etc/nginx/modsec/unicode.mapping 20127'. in /etc/nginx/nginx.conf:17

nginx: [emerg] "modsecurity_rules_file" directive Rules error. File: /etc/nginx/modsec/modsecurity.conf. Line: 192. Column: 17. Failed to load locate the unicode map file from: unicode.mapping 20127 Looking at: 'unicode.mapping 20127', 'unicode.mapping 20127', '/etc/nginx/modsec/unicode.mapping 20127', '/etc/nginx/modsec/unicode.mapping 20127'. in /etc/nginx/nginx.conf:17

To Reproduce

Steps to reproduce the behavior:

docker build on the Dockerfile. Add config values in setupEnvAndStart.sh for variables listed in the nginx.conf above docker run on resulting image (exposing port 443)

Expected behavior

Nginx should start up without errors, instead it errors as above

Server (please complete the following information):

• ModSecurity version (and connector): ModSecurity v3.0.2 (but pulled from v3/master at build time) with nginx-connector v1.0.0
• WebServer: nginx-1.15.3
• OS (and distro): RHEL 7.5

Rule Set (please complete the following information):

v3.0/master https://github.com/SpiderLabs/owasp-modsecurity-crs

Additional context

This has previously worked - on 23/10/18 this was built with image size of 451.11 MB...today's (25/10/18) build is 462.04 MB - no additional config was included from our end.

[zimmerle]

Hi @tinyoafman,

Thank you for the detailed report. The support for SecUnicodeMapFile was broken and partially fixed on v3/master. The file mentioned on the recommended configuration is available here: https://github.com/SpiderLabs/ModSecurity/blob/49495f1925a14f74f93cb0ef01172e5abc3e4c55/unicode.mapping#L7-L17

However, as you well noticed, it is not part of v3/master yet.

Yet, there is still an issue on SecUnicodeMapFile, as it is not separating the map code from the file name.

Three things that needs to be done before mark this issue as closed: 1 - Put the unicode.mapping as part of v3/master 2 - Fix SecUnicodeMapFile to better understand the map code (last parameter). 3 - Create a test case for this thing.

Further reading on SecUnicodeMapFile available here.

[victorhora]

1 - Put the unicode.mapping as part of v3/master

Added as of https://github.com/SpiderLabs/ModSecurity/commit/662fe63a47d012b89175ea7da5be1a8f7e76014c.

The error message "Failed to load locate the unicode map file" should go away as long as the file is on the same directory as the configuration file. Alternatively, the user can also change / disable the functionality on the modsecurity.conf-recommended.

2 - Fix SecUnicodeMapFile to better understand the map code (last parameter). 3 - Create a test case for this thing.

Working on it :)

[ghost]

same issue, and Even If the unicode.mapping file is placed inside the configuration folder, still the Nginx can't be started.

Nginx does detect the file unicode.mapping , but can't make use of it .

[victorhora]

@sp9ood

I can not reproduce the same issue. Are you sure you are running the latest commit from master? As of 662fe63 the unicode.mapping file is available on the v3/master branch and Nginx with libModSecurity should start fine.

The SecUnicodeMapFile directive using the syntax from modsecurity.conf-recommended should not cause errors if the file is present.

[victorhora]

Results of some testing with SecUnicodeMapFile:

Rules:

SecRule "ARGS" "@contains dop" "phase:2,id:210839,deny,log,auditlog,msg:BOOM,t:none,t:utf8toUnicode,t:urlDecodeUni,multiMatch"
SecUnicodeMapFile unicode.mapping-1251 1251

Content of unicode.mapping-1251 file:

1251 0434:64 043e:6f 0440:70

Request:

curl -v /?http://127.0.0.1:80/?a=дор

V2 debug logs without SecUnicodeMapFile specified:

18/Oct/2018:07:59:06 --0400] [127.0.0.1/sid#55811ca68c50][rid#55811cadd820][/][4] Recipe: Invoking rule 55811caa30c0; [file "/usr/local/nginx/conf/modsecurity.conf"] [line "296"] [id "210839"].
[18/Oct/2018:07:59:06 --0400] [127.0.0.1/sid#55811ca68c50][rid#55811cadd820][/][5] Rule 55811caa30c0: SecRule "ARGS" "@contains dop" "phase:2,id:210839,deny,log,auditlog,msg:BOOM,t:none,t:utf8toUnicode,t:urlDecodeUni,multiMatch"
[18/Oct/2018:07:59:06 --0400] [127.0.0.1/sid#55811ca68c50][rid#55811cadd820][/][4] Transformation completed in 1 usec.
[18/Oct/2018:07:59:06 --0400] [127.0.0.1/sid#55811ca68c50][rid#55811cadd820][/][4] Executing operator "contains" with param "dop" against ARGS:a.
[18/Oct/2018:07:59:06 --0400] [127.0.0.1/sid#55811ca68c50][rid#55811cadd820][/][9] Target value: "\xd0\xb4\xd0\xbe\xd1\x80"
[18/Oct/2018:07:59:06 --0400] [127.0.0.1/sid#55811ca68c50][rid#55811cadd820][/][4] Operator completed in 1 usec.
[18/Oct/2018:07:59:06 --0400] [127.0.0.1/sid#55811ca68c50][rid#55811cadd820][/][9] T (0) Utf8toUnicode: "%u0434%u043e%u0440"
[18/Oct/2018:07:59:06 --0400] [127.0.0.1/sid#55811ca68c50][rid#55811cadd820][/][4] Transformation completed in 24 usec.
[18/Oct/2018:07:59:06 --0400] [127.0.0.1/sid#55811ca68c50][rid#55811cadd820][/][4] Executing operator "contains" with param "dop" against ARGS:a.
[18/Oct/2018:07:59:06 --0400] [127.0.0.1/sid#55811ca68c50][rid#55811cadd820][/][9] Target value: "%u0434%u043e%u0440"
[18/Oct/2018:07:59:06 --0400] [127.0.0.1/sid#55811ca68c50][rid#55811cadd820][/][4] Operator completed in 1 usec.
[18/Oct/2018:07:59:06 --0400] [127.0.0.1/sid#55811ca68c50][rid#55811cadd820][/][9] T (0) urlDecodeUni: "4>@"
[18/Oct/2018:07:59:06 --0400] [127.0.0.1/sid#55811ca68c50][rid#55811cadd820][/][4] Transformation completed in 44 usec.
[18/Oct/2018:07:59:06 --0400] [127.0.0.1/sid#55811ca68c50][rid#55811cadd820][/][4] Executing operator "contains" with param "dop" against ARGS:a.
[18/Oct/2018:07:59:06 --0400] [127.0.0.1/sid#55811ca68c50][rid#55811cadd820][/][9] Target value: "4>@"
[18/Oct/2018:07:59:06 --0400] [127.0.0.1/sid#55811ca68c50][rid#55811cadd820][/][4] Operator completed in 0 usec.
[18/Oct/2018:07:59:06 --0400] [127.0.0.1/sid#55811ca68c50][rid#55811cadd820][/][4] Rule returned 0.

V2 debug logs with SecUnicodeMapFile specified:

[18/Oct/2018:08:53:30 --0400] [127.0.0.1/sid#55961dc86c50][rid#55961dcfb820][/][4] Recipe: Invoking rule 55961dcc12c8; [file "/usr/local/nginx/conf/modsecurity.conf"] [line "296"] [id "210839"].
[18/Oct/2018:08:53:30 --0400] [127.0.0.1/sid#55961dc86c50][rid#55961dcfb820][/][5] Rule 55961dcc12c8: SecRule "ARGS" "@contains dop" "phase:2,id:210839,deny,log,auditlog,msg:BOOM,t:none,t:utf8toUnicode,t:urlDecodeUni,multiMatch"
[18/Oct/2018:08:53:30 --0400] [127.0.0.1/sid#55961dc86c50][rid#55961dcfb820][/][4] Transformation completed in 1 usec.
[18/Oct/2018:08:53:30 --0400] [127.0.0.1/sid#55961dc86c50][rid#55961dcfb820][/][4] Executing operator "contains" with param "dop" against ARGS:a.
[18/Oct/2018:08:53:30 --0400] [127.0.0.1/sid#55961dc86c50][rid#55961dcfb820][/][9] Target value: "\xd0\xb4\xd0\xbe\xd1\x80"
[18/Oct/2018:08:53:30 --0400] [127.0.0.1/sid#55961dc86c50][rid#55961dcfb820][/][4] Operator completed in 1 usec.
[18/Oct/2018:08:53:30 --0400] [127.0.0.1/sid#55961dc86c50][rid#55961dcfb820][/][9] T (0) Utf8toUnicode: "%u0434%u043e%u0440"
[18/Oct/2018:08:53:30 --0400] [127.0.0.1/sid#55961dc86c50][rid#55961dcfb820][/][4] Transformation completed in 23 usec.
[18/Oct/2018:08:53:30 --0400] [127.0.0.1/sid#55961dc86c50][rid#55961dcfb820][/][4] Executing operator "contains" with param "dop" against ARGS:a.
[18/Oct/2018:08:53:30 --0400] [127.0.0.1/sid#55961dc86c50][rid#55961dcfb820][/][9] Target value: "%u0434%u043e%u0440"
[18/Oct/2018:08:53:30 --0400] [127.0.0.1/sid#55961dc86c50][rid#55961dcfb820][/][4] Operator completed in 0 usec.
[18/Oct/2018:08:53:30 --0400] [127.0.0.1/sid#55961dc86c50][rid#55961dcfb820][/][9] T (0) urlDecodeUni: "dop"
[18/Oct/2018:08:53:30 --0400] [127.0.0.1/sid#55961dc86c50][rid#55961dcfb820][/][4] Transformation completed in 43 usec.
[18/Oct/2018:08:53:30 --0400] [127.0.0.1/sid#55961dc86c50][rid#55961dcfb820][/][4] Executing operator "contains" with param "dop" against ARGS:a.
[18/Oct/2018:08:53:30 --0400] [127.0.0.1/sid#55961dc86c50][rid#55961dcfb820][/][9] Target value: "dop"
[18/Oct/2018:08:53:30 --0400] [127.0.0.1/sid#55961dc86c50][rid#55961dcfb820][/][4] Operator completed in 0 usec.
[18/Oct/2018:08:53:30 --0400] [127.0.0.1/sid#55961dc86c50][rid#55961dcfb820][/][4] Rule returned 1.

V3 debug logs with/without SecUnicodeMapFile specified:

[153986746362.341084] [/?a=дор] [4] (Rule: 210839) Executing operator "Contains" with param "dop" against ARGS.
[153986746362.341084] [/?a=дор] [9]  T (1) t:utf8toUnicode: "%u0434%u043e%u0440"
[153986746362.341084] [/?a=дор] [9]  T (2) t:urlDecodeUni: "4>@"
[153986746362.341084] [/?a=дор] [9] multiMatch is enabled. 3 values to be tested.
[153986746362.341084] [/?a=дор] [9] Target value: "\xd0\xb4\xd0\xbe\xd1\x80" (Variable: ARGS:a)
[153986746362.341084] [/?a=дор] [9] Target value: "%u0434%u043e%u0440" (Variable: ARGS:a)
[153986746362.341084] [/?a=дор] [9] Target value: "4>@" (Variable: ARGS:a)
[153986746362.341084] [/?a=дор] [4] Rule returned 0.
[153986746362.341084] [/?a=дор] [9] Matched vars cleaned.

Translating this into a test case for v3.

[nasatome]

same issue with v3/master: https://github.com/SpiderLabs/ModSecurity/commit/662fe63a47d012b89175ea7da5be1a8f7e76014c

[ghost]

@victorhora

First this Error

nginx: [emerg] "modsecurity_rules_file" directive Rules error. File: /etc/nginx/modsec/modsecurity.conf. Line: 234. Column: 17. Failed to load locate the unicode map file from: unicode.mapping 20127 Looking at: 'unicode.mapping 20127', 'unicode.mapping 20127', '/etc/nginx/modsec/unicode.mapping 20127', '/etc/nginx/modsec/unicode.mapping 20127'. in /etc/nginx/nginx.conf:28 nginx: configuration file /etc/nginx/nginx.conf test failed

Then $ sudo cp ~/ModSecurity/unicode.mapping /etc/nginx/modsec/

$ sudo nginx -t Segmentation fault

$systemctl status nginx.service ● nginx.service - nginx - high performance web server Loaded: loaded (/etc/systemd/system/nginx.service; enabled; vendor preset: enabled) Active: failed (Result: core-dump) since Mon 2018-10-29 06:13:10 UTC; 7s ago

Commenting out #SecUnicodeMapFile unicode.mapping 20127

Makes Everything Normal.

[Jumbledwords]

I'm encountering this issue as well, except not in a docker. I'm running a minimal install of Ubuntu server.

Shubham-Panwar describes exactly what I encountered.

[AnoopAlias]

ls /etc/nginx/conf.d/modsecurity.conf /etc/nginx/conf.d/unicode.mapping

/etc/nginx/conf.d/modsecurity.conf /etc/nginx/conf.d/unicode.mapping

This works ( I mean having modsecurity.conf and unicode.mapping in the same folder)

[ghost]

In my case , both files are at same location, But the Nginx throws Segmentation fault .

btw, I am using v3/master Mod-security on ubuntu server minimal.

[victorhora]

Can you folks tell us which version of the Nginx connector and Nginx itself you are using? If you could share at least part of your modsecurity.conf and nginx.conf files would be great :)

[ghost]

Nginx Version - 1.15.5 Modsec - v3/master Nginx connector- (I don't know version but I used git clone --depth 1 https://github.com/SpiderLabs/ModSecurity-nginx.git)

Here is the guide which I used

https://www.nginx.com/blog/compiling-and-installing-modsecurity-for-open-source-nginx/

[ghost]

Here are nginx.conf and modsecurity.conf files

nginx.txt

modsecurity.txt

[Jumbledwords]

I used the same guide, and got the same results.

[phnxfire]

I have the same issue as jumbledwords (i.e. in Ubuntu) and I followed the same guide as Shuban (https://www.nginx.com/blog/compiling-and-installing-modsecurity-for-open-source-nginx/).

[ghost]

Thanks a lot to Developers/Maintainers @zimmerle @victorhora , SecUnicodeMapFile unicode.mapping file is no-longer creating errors atleast in my setup

Ubuntu cosmic minimal Nginx 1.15.5 Modsec v3/master

kudos

[HankQuiter]

I am still seeing this issue in docker containers with alpine linux, nginx (1.15.5 and 1.15.3). I cannot second the statement that Shubham-Panwar just made about it not throwing errors anymore. I would also like to further the claim that this is not fixed by pointing out that the unicode.mapping file that is sought after by the configuration does not get created. I May be missing something - but a couple weeks ago building from v3/master worked with the same dockerfile (which clones the v3/master in the build). I see above others are talking about ways to get the unicode.mapping file in the right location or symlink it to the proper place - but this file does not exist in my built system and therefore cannot be found.

[ghost]

@HankQuiter I don't know much about your system,

but if you use $ git clone --recursive -b v3/master https://github.com/SpiderLabs/ModSecurity

there is unicode.mapping file inside ModSecurity Folder , Which you can copy to the configuration folder.

[gwo5002]

It appears that e3b9f7c9131b16a260044755a08edf94715bf705 broke libmodsecurity compilation for me. Output from make:

libtool: compile:  g++ -DHAVE_CONFIG_H -I. -std=c++11 -I.. -g -I../others -fPIC -O3 -I../headers -DWITH_GEOIP -I/usr/include -DWITH_YAJL -DPCRE_HAVE_JIT -I/usr/include/libxml2 -DWITH_LIBXML2 -g -O2 -MT libmodsecurity_la-rules_properties.lo -MD -MP -MF .deps/libmodsecurity_la-rules_properties.Tpo -c rules_properties.cc  -fPIC -DPIC -o .libs/libmodsecurity_la-rules_properties.o
rules_properties.cc: In static member function 'static void modsecurity::ConfigUnicodeMap::loadConfig(std::string, double, modsecurity::RulesProperties*, std::string*)':
rules_properties.cc:50:48: error: no match for 'operator[]' (operand types are 'std::shared_ptr<int []>' and 'int')
     driver->m_unicodeMapTable.m_unicodeMapTable[0x3002] = 0x2e;
                                                ^
rules_properties.cc:52:48: error: no match for 'operator[]' (operand types are 'std::shared_ptr<int []>' and 'int')
     driver->m_unicodeMapTable.m_unicodeMapTable[0xff61] = 0x2e;
                                                ^
rules_properties.cc:54:48: error: no match for 'operator[]' (operand types are 'std::shared_ptr<int []>' and 'int')
     driver->m_unicodeMapTable.m_unicodeMapTable[0xff0e] = 0x2e;
                                                ^
rules_properties.cc:56:48: error: no match for 'operator[]' (operand types are 'std::shared_ptr<int []>' and 'int')
     driver->m_unicodeMapTable.m_unicodeMapTable[0x002e] = 0x2e;
                                                ^
rules_properties.cc:109:64: error: no match for 'operator[]' (operand types are 'std::shared_ptr<int []>' and 'int')
                     driver->m_unicodeMapTable.m_unicodeMapTable[code] = Map;
                                                                ^
In file included from /usr/include/c++/4.8.5/bits/shared_ptr.h:52:0,
                 from /usr/include/c++/4.8.5/memory:82,
                 from ../headers/modsecurity/modsecurity.h:80,
                 from ../headers/modsecurity/rules_properties.h:32,
                 from rules_properties.cc:16:
/usr/include/c++/4.8.5/bits/shared_ptr_base.h: In instantiation of 'std::__shared_ptr<_Tp, _Lp>::__shared_ptr(std::unique_ptr<_Up, _Ep>&&) [with _Tp1 = int []; _Del = std::default_delete<int []>; _Tp = int []; __gnu_cxx::_Lock_policy _Lp = (__gnu_cxx::_Lock_policy)2u]':
/usr/include/c++/4.8.5/bits/shared_ptr_base.h:876:4:   required from 'std::__shared_ptr<_Tp, _Lp>& std::__shared_ptr<_Tp, _Lp>::operator=(std::unique_ptr<_Up, _Ep>&&) [with _Tp1 = int []; _Del = std::default_delete<int []>; _Tp = int []; __gnu_cxx::_Lock_policy _Lp = (__gnu_cxx::_Lock_policy)2u]'
/usr/include/c++/4.8.5/bits/shared_ptr.h:307:4:   required from 'std::shared_ptr<_Tp>& std::shared_ptr<_Tp>::operator=(std::unique_ptr<_Up, _Ep>&&) [with _Tp1 = int []; _Del = std::default_delete<int []>; _Tp = int []]'
rules_properties.cc:44:49:   required from here
/usr/include/c++/4.8.5/bits/shared_ptr_base.h:819:35: error: cannot convert 'std::unique_ptr<int []>::pointer {aka int*}' to 'int (*)[]' in initialization
  : _M_ptr(__r.get()), _M_refcount()
                                   ^

FWIW I'm on Amazon Linux and can still build on 84ece3edcba162fcea37a11588a99890ef92293e

[zimmerle]

It appears that e3b9f7c broke libmodsecurity compilation for me. Output from make:

libtool: compile:  g++ -DHAVE_CONFIG_H -I. -std=c++11 -I.. -g -I../others -fPIC -O3 -I../headers -DWITH_GEOIP -I/usr/include -DWITH_YAJL -DPCRE_HAVE_JIT -I/usr/include/libxml2 -DWITH_LIBXML2 -g -O2 -MT libmodsecurity_la-rules_properties.lo -MD -MP -MF .deps/libmodsecurity_la-rules_properties.Tpo -c rules_properties.cc  -fPIC -DPIC -o .libs/libmodsecurity_la-rules_properties.o
rules_properties.cc: In static member function 'static void modsecurity::ConfigUnicodeMap::loadConfig(std::string, double, modsecurity::RulesProperties*, std::string*)':
rules_properties.cc:50:48: error: no match for 'operator[]' (operand types are 'std::shared_ptr<int []>' and 'int')
     driver->m_unicodeMapTable.m_unicodeMapTable[0x3002] = 0x2e;
                                                ^
rules_properties.cc:52:48: error: no match for 'operator[]' (operand types are 'std::shared_ptr<int []>' and 'int')
     driver->m_unicodeMapTable.m_unicodeMapTable[0xff61] = 0x2e;
                                                ^
rules_properties.cc:54:48: error: no match for 'operator[]' (operand types are 'std::shared_ptr<int []>' and 'int')
     driver->m_unicodeMapTable.m_unicodeMapTable[0xff0e] = 0x2e;
                                                ^
rules_properties.cc:56:48: error: no match for 'operator[]' (operand types are 'std::shared_ptr<int []>' and 'int')
     driver->m_unicodeMapTable.m_unicodeMapTable[0x002e] = 0x2e;
                                                ^
rules_properties.cc:109:64: error: no match for 'operator[]' (operand types are 'std::shared_ptr<int []>' and 'int')
                     driver->m_unicodeMapTable.m_unicodeMapTable[code] = Map;
                                                                ^
In file included from /usr/include/c++/4.8.5/bits/shared_ptr.h:52:0,
                 from /usr/include/c++/4.8.5/memory:82,
                 from ../headers/modsecurity/modsecurity.h:80,
                 from ../headers/modsecurity/rules_properties.h:32,
                 from rules_properties.cc:16:
/usr/include/c++/4.8.5/bits/shared_ptr_base.h: In instantiation of 'std::__shared_ptr<_Tp, _Lp>::__shared_ptr(std::unique_ptr<_Up, _Ep>&&) [with _Tp1 = int []; _Del = std::default_delete<int []>; _Tp = int []; __gnu_cxx::_Lock_policy _Lp = (__gnu_cxx::_Lock_policy)2u]':
/usr/include/c++/4.8.5/bits/shared_ptr_base.h:876:4:   required from 'std::__shared_ptr<_Tp, _Lp>& std::__shared_ptr<_Tp, _Lp>::operator=(std::unique_ptr<_Up, _Ep>&&) [with _Tp1 = int []; _Del = std::default_delete<int []>; _Tp = int []; __gnu_cxx::_Lock_policy _Lp = (__gnu_cxx::_Lock_policy)2u]'
/usr/include/c++/4.8.5/bits/shared_ptr.h:307:4:   required from 'std::shared_ptr<_Tp>& std::shared_ptr<_Tp>::operator=(std::unique_ptr<_Up, _Ep>&&) [with _Tp1 = int []; _Del = std::default_delete<int []>; _Tp = int []]'
rules_properties.cc:44:49:   required from here
/usr/include/c++/4.8.5/bits/shared_ptr_base.h:819:35: error: cannot convert 'std::unique_ptr<int []>::pointer {aka int*}' to 'int (*)[]' in initialization
  : _M_ptr(__r.get()), _M_refcount()
                                   ^

FWIW I'm on Amazon Linux and can still build on 84ece3e

Fixed at: 18cdffdbca75e6b9f790f6df2807a32cd805c0a0

[kavvalos]

I had an issue after update nginx

Nov 21 23:22:49 vpbx.voipcom.gr nginx[19879]: nginx: [emerg] "modsecurity_rules_file" directive Rules error. File: /etc/nginx/modsec/modsecurity.conf. Line: 236. Column: 17. Failed to open the unicode map file from Nov 21 23:22:49 vpbx.voipcom.gr systemd[1]: nginx.service: control process exited, code=exited status=1 Nov 21 23:22:49 vpbx.voipcom.gr systemd[1]: Failed to start nginx - high performance web server.

I copied unicode.mapping in the same folder of modsecurity.conf and I change the next line SecUnicodeMapFile unicode.mapping 20127 to ==>> SecUnicodeMapFile ./unicode.mapping 20127

nginx started

I hope this help someone

System: Debian 8

[victorhora]

Hey @kavvalos

Thanks for your report.

I'm surprised that you stumbled on this. Adding the "./" to the file name shouldn't be necessary. Can you confirm that if you remove this libModSecurity is unable to open the file?

Also, commit https://github.com/SpiderLabs/ModSecurity/commit/662fe63a47d012b89175ea7da5be1a8f7e76014c should add the unicode.mapping to your setup so that you don't need to copy it manually from somewhere else.

[kavvalos]

If I put fulll path or I remove "./" nginx does not start

[victorhora]

Reopening this one for further investigation.

[rlovell3]

I just got involved with modsecurity today. Went through the Nginx guide to compiling/installing just like several people above. Had the same nginx -t config failures which quickly led me here. It seems I have moved past the fail by manually downloading the unicode.mapping file and placing it in the /etc/nginx/modsec directory. Subsequent nginx -t test passes. Running Ubuntu 18.04.1 on Digital Ocean droplet, nginx 1.15.7 with whatever the pull version is of modsecurity in the Nginx howto document. All compilation done today. As a humble outsider looking in, it seems simply that the modsec pull request I followed from the Nginx manual didn't have the unicode file. Now that nginx isn't complaining I'm going to drive on. Appreciate you guys putting together such a useful piece of kit.

[n0sign4l]

I get this error using openresty sbin/nginx -c conf/nginx.conf -t nginx: [emerg] "modsecurity_rules_file" directive Rules error. File: /usr/local/openresty/nginx/conf/modsecurity.conf. Line: 236. Column: 17. Failed to locate the unicode map file from: unicode.mapping Looking at: 'unicode.mapping', 'unicode.mapping', '/usr/local/openresty/nginx/conf/unicode.mapping', '/usr/local/openresty/nginx/conf/unicode.mapping'. in /usr/local/openresty/nginx/conf/nginx.conf:48 nginx: configuration file /usr/local/openresty/nginx/conf/nginx.conf test failed

[n0sign4l]

Hey, any news on this issue? How can I solve it? @zimmerle @victorhora

[HankQuiter]

Hey, any news on this issue? How can I solve it? @zimmerle @victorhora

see above shubham-panwar said

@HankQuiter I don't know much about your system,

but if you use $ git clone --recursive -b v3/master https://github.com/SpiderLabs/ModSecurity

there is unicode.mapping file inside ModSecurity Folder , Which you can copy to the configuration folder.

[rlovell3]

Check ownership/permissions at the file level and also the directory level.

Manually retrieve the unicode.mapping file from the repository.
Store in /etc/nginx/modsec/unicode.mapping.
Duplicate the ownership/permissions of your modsecurity.conf file. In my case that is: root:root, 644.

If you have the unicode.mapping file and the modsecurity.conf file in the same directory, confirm the following directive in your modsecurity.conf file:

SecUnicodeMapFile unicode.mapping 20127

[pakjiddat]

I was able to fix the problem by copying the unicode.mapping file to /etc/nginx/modsec

[dvdknaap]

copying the unicode.mapping doesn't work for me so i have to disable it :(

[hello-yunshu]

I had an issue after update nginx

Nov 21 23:22:49 vpbx.voipcom.gr nginx[19879]: nginx: [emerg] "modsecurity_rules_file" directive Rules error. File: /etc/nginx/modsec/modsecurity.conf. Line: 236. Column: 17. Failed to open the unicode map file from Nov 21 23:22:49 vpbx.voipcom.gr systemd[1]: nginx.service: control process exited, code=exited status=1 Nov 21 23:22:49 vpbx.voipcom.gr systemd[1]: Failed to start nginx - high performance web server.

I copied unicode.mapping in the same folder of modsecurity.conf and I change the next line SecUnicodeMapFile unicode.mapping 20127 to ==>> SecUnicodeMapFile ./unicode.mapping 20127

nginx started

I hope this help someone

System: Debian 8

This save my problem. Thanks

[pl853]

If it's worth anything. I fixed the problem by copying the unicode.mapping file to the nginx modsec folder.

sudo cp path/to/cloned/modsec/repo/unicode.mapping /etc/nginx/modsec/unicode.mapping

[m-faraz]

Started this using this URL, nginx 1.16.1 using ppa:nginx/stable

Encountered same problem, just did cp /opt/ModSecurity/unicode.mapping /etc/nginx/modsec/ and everything worked. Just need to document this now, it seems!

[FortisFortuna]

cp /opt/ModSecurity/unicode.mapping /etc/nginx/modsec/

This fixed it for me

[stellarpower]

This file (along with others) are included in the automake here but don't seem to be copied over as part of a standard configure, make, make install process. Would it be possible to have one of the following?

• Include copying these files in install process as part of default recipe
• Add a make target that explicitly installs these files in the destination dir
• Document how this should be done correctly in a section of the wiki on Compilation recipes:

Unless I'm missing something and there's already an obvious way (I tend to prefer CMake for compiled projects), but it seems to be copied manually in this recipe for CentOS unless this is out of date. I'm also blindly obeying the errors until they go away and not spent time into looking into what this file is actually doing, for reference ;)