Nginx Segfaulted

[xerudro]

Hi, yesterday i decided to try to integrate Imunify360 solution to my VPS with HestiaCP, installed on Debian11 , with NGINX webserver. Because the server was a non-supported version i had to make the configs for integrations and preparations myself, before installing and using Imunify360. As a requirement from the Cloudlinux devs, the ModSecurity had to be installed and configured with clean settings, before the installation of the antivir and firewall solution. I asked them if they can offer me a guide, or atleast something to start from and they offered me this link : https://cloudlinux.zendesk.com/hc/en-us/articles/360018872859-How-to-install-ModSecurity-on-the-nginx-only-server-?source=search, but this is for Almalinux or CentOS, so i searched a google similar one for debian and i found one but for debian10, not 11 and i`ve tried to adapt both instructions from what i found with the one provided by the guys from Cloudlinux... and after few errors on git submodule init and submodule update, and searches on google, for fixes and workarounds "i managed" to finish the modsecurity installation. I started the installation and watched the logs for malware or attacks on the server and i noticed this errors logged by Imunify : flash kernel: [27774.579002] nginx[113980]: segfault at 55aaa76a2f50 ip 00007fb486356c35 sp 00007ffc372ed440 error 4. Searching in the syslog i found this errors : Reloaded nginx - high performance web server. Jul 8 07:48:15 flash kernel: [27774.074330] nginx[113820]: segfault at 55aaa7555f00 ip 00007fb486356c35 sp 00007ffc372ed440 error 4 in libc-2.31.so[7fb48633e000+159000] Jul 8 07:48:15 flash kernel: [27774.076600] Code: fe ff 49 83 c5 02 41 0f b7 6d fe 49 89 c6 4c 8d 78 fe 4d 85 e4 75 13 eb cc 0f 1f 40 00 4c 8b 63 08 48 83 c3 08 4d 85 e4 74 bb <66> 41 3b 2c 24 75 ec 49 8d 7c 24 02 4c 89 fa 4c 89 ee e8 04 76 fe Jul 8 07:48:15 flash kernel: [27774.093507] nginx[113824]: segfault at 55aaa7555f00 ip 00007fb486356c35 sp 00007ffc372ed440 error 4 Jul 8 07:48:15 flash kernel: [27774.093509] nginx[113822]: segfault at 55aaa7555f00 ip 00007fb486356c35 sp 00007ffc372ed440 error 4 in libc-2.31.so[7fb48633e000+159000] Jul 8 07:48:15 flash kernel: [27774.095361] in libc-2.31.so[7fb48633e000+159000] Jul 8 07:48:15 flash kernel: [27774.097076] nginx[113823]: segfault at 55aaa7555f00 ip 00007fb486356c35 sp 00007ffc372ed440 error 4 in libc-2.31.so[7fb48633e000+159000] Jul 8 07:48:15 flash kernel: [27774.097081] Code: fe ff 49 83 c5 02 41 0f b7 6d fe 49 89 c6 4c 8d 78 fe 4d 85 e4 75 13 eb cc 0f 1f 40 00 4c 8b 63 08 48 83 c3 08 4d 85 e4 74 bb <66> 41 3b 2c 24 75 ec 49 8d 7c 24 02 4c 89 fa 4c 89 ee e8 04 76 fe Jul 8 07:48:15 flash kernel: [27774.097746] Code: fe ff 49 83 c5 02 41 0f b7 6d fe 49 89 c6 4c 8d 78 fe 4d 85 e4 75 13 eb cc 0f 1f 40 00 4c 8b 63 08 48 83 c3 08 4d 85 e4 74 bb <66> 41 3b 2c 24 75 ec 49 8d 7c 24 02 4c 89 fa 4c 89 ee e8 04 76 fe Jul 8 07:48:15 flash kernel: [27774.098785] Jul 8 07:48:15 flash kernel: [27774.106411] nginx[113818]: segfault at 55aaa7555f00 ip 00007fb486356c35 sp 00007ffc372ed440 error 4 Jul 8 07:48:15 flash kernel: [27774.107125] nginx[113821]: segfault at 55aaa7555f00 ip 00007fb486356c35 sp 00007ffc372ed440 error 4 in libc-2.31.so[7fb48633e000+159000] Jul 8 07:48:15 flash kernel: [27774.107131] Code: fe ff 49 83 c5 02 41 0f b7 6d fe 49 89 c6 4c 8d 78 fe 4d 85 e4 75 13 eb cc 0f 1f 40 00 4c 8b 63 08 48 83 c3 08 4d 85 e4 74 bb <66> 41 3b 2c 24 75 ec 49 8d 7c 24 02 4c 89 fa 4c 89 ee e8 04 76 fe Jul 8 07:48:15 flash kernel: [27774.107214] Code: fe ff 49 83 c5 02 41 0f b7 6d fe 49 89 c6 4c 8d 78 fe 4d 85 e4 75 13 eb cc 0f 1f 40 00 4c 8b 63 08 48 83 c3 08 4d 85 e4 74 bb <66> 41 3b 2c 24 75 ec 49 8d 7c 24 02 4c 89 fa 4c 89 ee e8 04 76 fe Jul 8 07:48:15 flash kernel: [27774.108524] in libc-2.31.so[7fb48633e000+159000] Jul 8 07:48:15 flash kernel: [27774.117463] Code: fe ff 49 83 c5 02 41 0f b7 6d fe 49 89 c6 4c 8d 78 fe 4d 85 e4 75 13 eb cc 0f 1f 40 00 4c 8b 63 08 48 83 c3 08 4d 85 e4 74 bb <66> 41 3b 2c 24 75 ec 49 8d 7c 24 02 4c 89 fa 4c 89 ee e8 04 76 fe Jul 8 07:48:15 flash kernel: [27774.132899] nginx[113819]: segfault at 55aaa7555f00 ip 00007fb486356c35 sp 00007ffc372ed440 error 4 in libc-2.31.so[7fb48633e000+159000] Jul 8 07:48:15 flash kernel: [27774.135538] Code: fe ff 49 83 c5 02 41 0f b7 6d fe 49 89 c6 4c 8d 78 fe 4d 85 e4 75 13 eb cc 0f 1f 40 00 4c 8b 63 08 48 83 c3 08 4d 85 e4 74 bb <66> 41 3b 2c 24 75 ec 49 8d 7c 24 02 4c 89 fa 4c 89 ee e8 04 76 fe Jul 8 07:48:15 flash kernel: [27774.138279] nginx[113825]: segfault at 55aaa7555f00 ip 00007fb486356c35 sp 00007ffc372ed440 error 4 in libc-2.31.so[7fb48633e000+159000] Jul 8 07:48:15 flash kernel: [27774.138286] Code: fe ff 49 83 c5 02 41 0f b7 6d fe 49 89 c6 4c 8d 78 fe 4d 85 e4 75 13 eb cc 0f 1f 40 00 4c 8b 63 08 48 83 c3 08 4d 85 e4 74 bb <66> 41 3b 2c 24 75 ec 49 8d 7c 24 02 4c 89 fa 4c 89 ee e8 04 76 fe Jul 8 07:48:16 flash kernel: [27774.577842] nginx[113981]: segfault at 55aaa76a2f50 ip 00007fb486356c35 sp 00007ffc372ed440 error 4 in libc-2.31.so[7fb48633e000+159000] Jul 8 07:48:16 flash kernel: [27774.579002] nginx[113980]: segfault at 55aaa76a2f50 ip 00007fb486356c35 sp 00007ffc372ed440 error 4 Jul 8 07:48:16 flash kernel: [27774.580153] Code: fe ff 49 83 c5 02 41 0f b7 6d fe 49 89 c6 4c 8d 78 fe 4d 85 e4 75 13 eb cc 0f 1f 40 00 4c 8b 63 08 48 83 c3 08 4d 85 e4 74 bb <66> 41 3b 2c 24 75 ec 49 8d 7c 24 02 4c 89 fa 4c 89 ee e8 04 76 fe Jul 8 07:48:16 flash kernel: [27774.580155] in libc-2.31.so[7fb48633e000+159000] Jul 8 07:48:16 flash kernel: [27774.585765] Code: fe ff 49 83 c5 02 41 0f b7 6d fe 49 89 c6 4c 8d 78 fe 4d 85 e4 75 13 eb cc 0f 1f 40 00 4c 8b 63 08 48 83 c3 08 4d 85 e4 74 bb <66> 41 3b 2c 24 75 ec 49 8d 7c 24 02 4c 89 fa 4c 89 ee e8 04 76 fe

I searched on google and i`ve found that they are related to ModSecurity modules... Can you guys please help me fix them ?

[martinhsv]

Hello @xerudro ,

You haven't specified exactly what leads you to believe ModSecurity is involved with your issue. That certainly could be the case, but nothing in the output that you have provided suggests that (as far as I can see).

What versions are you using of ModSecurity, ModSecurity-nginx, and nginx are you using?

I can't really do anything with the output that you have provided. If you can provide a stack trace, that might provide some useful information.

In general, some types of issues like that can be caused by incompatibilities -- one such is if you aren't using the same pcre (pcre1 vs. pcre2) in all of those components.

[xerudro]

Hi. I am using nginx 1.25.1 but i want to install it on a Debian 11, or 12 ( if possible i want to stick with 12) if not i will revert to 11. I intend to use it with HestiaCP, and imunify360... or if you can suggest a better option except CSF. I don`t like the GUI of CSF. ... I have an VPS where i host few personal sites, and i have 2 clients ( friends ) who host their sites too. Can you please help me with a guide on how to compile it without any issues ? I tried different tuts to test and i had same nginx segfaults errors ... I am so tired of tring to make it work ...

[martinhsv]

Hello @xerudro ,

You didn't mention anything about the other pieces of information that I highlighted.

nginx 1.25.1 uses pcre2 by default. In this case, you need to be using at least v1.0.3 of the ModSecurity-nginx connector,

And your ModSecurity (>= v3.0.7) needs to be built with pcre2 (--with-pcre2) during the configure step.

[xerudro]

@martinhsv Sorry i didnt because now the server is a clean server. I just rebuilt the server, so the modsecurity and modsecurity-nginx are no more installed on the server. And why i am thinking the modsecurity is the cause is because i didnt had those problems before i tried to install and complile modsecurity. I also tried imunify on directadmin without any issus of that kind, but the support of the panel really sucks. Thats why i decided to try the self-hosted imunify...

[xerudro]

Can you please help me with a well documented guide how to install and compile it ? I am not a very techy guy. I was thinking to ask if you can do it for me... but i think it`s inappropriate.

[martinhsv]

For the build, if all you are building yourself is libModSecurity, there are compile recipes here: https://github.com/SpiderLabs/ModSecurity/wiki/Compilation-recipes-for-v3.x

For debian, you should be able to use the one for Ubuntu: https://github.com/SpiderLabs/ModSecurity/wiki/Compilation-recipes-for-v3.x#ubuntu-1804

The main exceptions in your case are likely, that:

• at the first step (the install stage), you need to include: libpcre2-dev and pcre2-utils
• as mentioned previously, at the ./configure step you need to specify ./configure --with-pcre2

[xerudro]

Hi again, Martin. the libpcre++-dev can be removed from the apt-get line? Because of this : E: Unable to locate package libpcre++-dev E: Couldn't find any package by regex 'libpcre++-dev'

[airween]

For Debian 11 you can try our repository for Debian and Ubuntu distributions:

https://modsecurity.digitalwave.hu

It contains both libmodsecurity3 and the connector for Nginx - but we support the Debian's Nginx.

[xerudro]

@airween this will work for Debian 12 also ? I tend to stuck with it if possible. If it is,does requires some extra steps and configurations except the ones from the page you provided me ?

[airween]

@airween this will work for Debian 12 also?

please read the page:

Supported Linux distribution(s)

• Debian 12 a.k.a. Bookworm, amd64
• Debian 11 a.k.a. Bullseye, amd64
• Debian 10 a.k.a. Buster, amd64
• Ubuntu 22.04 (LTS) a.k.a. Jammy, amd64
• Ubuntu 20.04 (LTS) a.k.a. Focal, amd64
• Ubuntu 18.04 (LTS) a.k.a. Bionic, amd64

I tend to stuck with it if possible. If it is,does requires some extra steps and configurations except the ones from the page you provided me ?

No, all necessary steps are on the site (including setting of the repository and configuring the server and the engine).

[xerudro]

Thanks, but i still have an question mark, ive seen on the page that the version of the nginx is 1.18, on the server its 1.25.1 that won`t create uncompatibility issues ?

[xerudro]

@airween i`ve tried to run the steps... and here is what i got : apt-cache policy libnginx-mod-http-modsecurity libmodsecurity3 modsecurity-crs libnginx-mod-http-modsecurity: Installed: (none) Candidate: 1.0.4-1~pre1+0~20230609~bpo12+d59e4ad1 Version table: 1.0.4-1~pre1+0~20230609~bpo12+d59e4ad1 900 500 http://modsecurity.digitalwave.hu/debian bookworm-backports/main amd64 Packages 1.0.3-1+b1 500 500 http://deb.debian.org/debian bookworm/main amd64 Packages 500 http://mirror.hetzner.com/debian/packages bookworm/main amd64 Packages libmodsecurity3: Installed: 3.0.10-1~pre1+0~20230609~bpo12+205dac0e Candidate: 3.0.10-1~pre1+0~20230609~bpo12+205dac0e Version table: *** 3.0.10-1~pre1+0~20230609~bpo12+205dac0e 900 500 http://modsecurity.digitalwave.hu/debian bookworm-backports/main amd64 Packages 100 /var/lib/dpkg/status 3.0.9-1 500 500 http://deb.debian.org/debian bookworm/main amd64 Packages 500 http://mirror.hetzner.com/debian/packages bookworm/main amd64 Packages modsecurity-crs: Installed: (none) Candidate: 3.3.4-1~bpo12+1 Version table: 3.3.4-1 500 500 http://deb.debian.org/debian bookworm/main amd64 Packages 500 http://mirror.hetzner.com/debian/packages bookworm/main amd64 Packages 3.3.4-1~bpo12+1 900 500 http://modsecurity.digitalwave.hu/debian bookworm-backports/main amd64 Packages root@flash:~# apt install libnginx-mod-http-modsecurity Reading package lists... Done Building dependency tree... Done Reading state information... Done Some packages could not be installed. This may mean that you have requested an impossible situation or if you are using the unstable distribution that some required packages have not yet been created or been moved out of Incoming. The following information may help to resolve the situation:

The following packages have unmet dependencies: libnginx-mod-http-modsecurity : Depends: libnginx-mod-http-ndk but it is not going to be installed Depends: nginx-abi-1.22.1-7 E: Unable to correct problems, you have held broken packages.

[airween]

Thanks, but i still have an question mark, ive seen on the page that the version of the nginx is 1.18, on the server its 1.25.1 that won`t create uncompatibility issues ?

Nginx version 1.18 is for Debian 11. Debian 12 contains Nginx 1.22.

The provided packages are compatible with the Debian packages from the official upstream.

[airween]

@xerudro - could you reformat this post? There are many strikethrough text, a bit hard to understand. And please use the code blocks to show the commands and outputs.

The following packages have unmet dependencies: libnginx-mod-http-modsecurity : Depends: libnginx-mod-http-ndk but it is not going to be installed Depends: nginx-abi-1.22.1-7 E: Unable to correct problems, you have held broken packages.

I have no idea where have you got that package, but my current packages do not depend on nginx-abi (yet). Both in official Debian repository and in Digitalwave's repo.

I assume that came from the third repository (http://mirror.hetzner.com/debian/packages), but actually I can't reach now that to check it.

Perhaps you should remove that from your sources list.

[xerudro]

Thanks @airween i will rebuild the VPS and remove that repo from sources list, try again and get back to you if it worked.

[xerudro]

It didnt work @airween, and i think i found the problem... the digitalwave repo uses nginx 1.22, and the panel installed on the server uses nginx 1.25.1... the repo with the culprit it`s hestia repo... because they use a newer version of nginx. I tried to install modsecurity from the digiralwave repo on a clean server, then install hestia on top of it, and the nginx was unable to restart for the panel installation... and was unable to create the panel subdomain due to a different version of nginx.

[airween]

the digitalwave repo uses nginx 1.22, and the panel installed on the server uses nginx 1.25.1...

Yes, that won't work. We provide 3rd-party packages for stable systems with official packages.

[xerudro]

So we get back to @martinhsv. Your solution will work with nginx 1.25.1 on debian 12 ? I havent tested it, i have seen the response from @airween, and i decided to test it first... because was a bit easier ... :)

[xerudro]

Ive rebuild the server to ubuntu 22.04, same nginx 1.25.1 version used, i think its inside the HestiaCP repository, cloned the git repository as on the ubuntu 22 documentation you sent me @martinhsv, and i noticed some WARNING messages...

Sorry for keep bothering you, but i am trying to learn first how to handle this things... and how to solve them :

configure: MaxMind library was not found configure: SSDEEP library was not found configure: LUA library was not found configure: WARNING: doxygen not found - will not generate any doxygen documentation Are this a worry motive or will be solved with nginx-connector ?

[xerudro]

@martinhsv @airween , i switched to ubuntu 22.10 and following the https://github.com/SpiderLabs/ModSecurity/wiki/Compilation-recipes-for-v3.x#ubuntu-2210, i got this warnings on compile... configure.ac:106: warning: The macroAC_TRY_COMPILE' is obsolete. configure.ac:106: You should run autoupdate. ./lib/autoconf/general.m4:2847: AC_TRY_COMPILE is expanded from... build/lua.m4:118: CHECK_FOR_LUA_AT is expanded from... build/lua.m4:5: CHECK_LUA is expanded from... configure.ac:106: the top level configure.ac:129: warning: The macro AC_TRY_LINK' is obsolete. configure.ac:129: You should run autoupdate. ./lib/autoconf/general.m4:2920: AC_TRY_LINK is expanded from... build/pcre.m4:15: CHECK_PCRE is expanded from... configure.ac:129: the top level configure.ac:140: warning: The macroAC_HEADER_STDC' is obsolete. configure.ac:140: You should run autoupdate. ./lib/autoconf/headers.m4:704: AC_HEADER_STDC is expanded from... configure.ac:140: the top level configure.ac:307: warning: AC_PROG_LEX without either yywrap or noyywrap is obsolete ./lib/autoconf/programs.m4:716: _AC_PROG_LEX is expanded from... ./lib/autoconf/programs.m4:709: AC_PROG_LEX is expanded from... configure.ac:307: the top level configure.ac:50: installing './ar-lib' configure.ac:50: installing './compile' configure.ac:147: installing './config.guess' configure.ac:147: installing './config.sub' configure.ac:45: installing './install-sh' configure.ac:45: installing './missing' parallel-tests: installing './test-driver' examples/multiprocess_c/Makefile.am: installing './depcomp' configure.ac: installing './ylwrap' configure.ac:106: warning: The macro AC_TRY_COMPILE' is obsolete. configure.ac:106: You should run autoupdate. ./lib/autoconf/general.m4:2847: AC_TRY_COMPILE is expanded from... build/lua.m4:118: CHECK_FOR_LUA_AT is expanded from... build/lua.m4:5: CHECK_LUA is expanded from... configure.ac:106: the top level configure.ac:129: warning: The macroAC_TRY_LINK' is obsolete. configure.ac:129: You should run autoupdate. ./lib/autoconf/general.m4:2920: AC_TRY_LINK is expanded from... build/pcre.m4:15: CHECK_PCRE is expanded from... configure.ac:129: the top level configure.ac:140: warning: The macro AC_HEADER_STDC' is obsolete. configure.ac:140: You should run autoupdate. ./lib/autoconf/headers.m4:704: AC_HEADER_STDC is expanded from... configure.ac:140: the top level configure.ac:307: warning: AC_PROG_LEX without either yywrap or noyywrap is obsolete ./lib/autoconf/programs.m4:716: _AC_PROG_LEX is expanded from... ./lib/autoconf/programs.m4:709: AC_PROG_LEX is expanded from... configure.ac:307: the top level Should i worry about them ?

And if you please help me with a guide on how to configure it with ngx_http_modsecurity_module.so on ubuntu 22.10, please ?

[xerudro]

I am using nginx 1.25.1 as i got some problems with the panel, trying to change nginx version to 1.22.

[xerudro]

*Sorry my bad, the distro is 22.04.6 LTS.

[martinhsv]

Regarding the software not found (lua, etc.): they indicate that you did not install all of the software dependencies (like the ones on the first line of the Ubuntu 22.10 recipe to which you linked).

The messages regarding 'obsolete' autoconf constructs: autoconf has been deprecating some features (although still supported for now). I have not encountered those myself, but you may be using a more recent version. As they are marked as warnings, I wouldn't expect any difficulties due to them.

As with my previous comment ( https://github.com/SpiderLabs/ModSecurity-nginx/issues/307#issuecomment-1629674985 ), if if you use that version (1.22) of nginx, it uses pcre2 by default os you need to do the ModSecurity configure step with '--with-pcre2'.

[martinhsv]

Anything further on this?