search

vel21ripn / nDPI

Open Source Deep Packet Inspection Software Toolkit
http://www.ntop.org
GNU Lesser General Public License v3.0
119 stars 59 forks source link

Kernel panic на ядре 4.9 #160

Closed omikhaylov closed 1 year ago

omikhaylov commented 1 year ago

Описание

Ядро 4.9 валится при выполнении iptables -A FORWARD -m ndpi --proto tiktok -j ACCEPT

nDPI Environment (please complete the following information):

Воспроизведение

Собранные заранее файлы были помещены в соответствующие директории:

Дополнительный контекст

Сборка проводилась в Docker. Использовалcя пакет linux-headers. Сборка из коммита 3ff5a9b работала исправно.

vel21ripn commented 1 year ago

В 4.9.189 mm/vmalloc.c:1375 BUG_ON(in_interrupt()); IMHO xt_check_match никогда не выполняется во время прерывания. Если сделать исправление

--- main.c  2023-02-17 12:35:59.247027489 +0300
+++ main.c.vmalloc  2023-03-13 23:52:52.020712420 +0300
@@ -595,7 +595,7 @@

 static void *malloc_wrapper(size_t size)
 {
-   if(size > 32*1024) {
+   if(0 && size > 32*1024) {
        /*
         * Workarround for 32bit systems
         * Large memory areas (more than 128 KB) are requested

баг проявится?

omikhaylov commented 1 year ago

После исправления баг не проявился.

vel21ripn commented 1 year ago

В ближайшее время сделаю более корректное изменение.

BrainSlayer commented 1 year ago

you need to refine your code. the check for 32kb is wrong. often kernels are patched with modified memory configurations and it might also be architecture dependend and allocator depenend (SLAB, SLUB etc). in addition you check for 32 kb not 32mb which is very small and unneccessary. kmalloc can safly allocate up to KMALLOC_MAX_SIZE (default is 32mb on slab. and 4mb for slub). but your malloc wrapper may also return NULL in case a large chunk is allocated using GFP_ATOMIC since this is a time critical allocation.

so my advise. check for KMALLOC_MAX_SIZE. so standard kernel kmalloc will be used in most cases. and maybe think about handling failed allocations for gfp_atomic if there is such a case

vel21ripn commented 1 year ago

@BrainSlayer Thanks for the hint about KMALLOC_MAX_SIZE. This code was a temporary workaround for a memory allocation problem in the kernel for some non-X86 architecture. There was a problem to allocate 128kB of RAM. For 2019, during the operation of the module, it was not required to allocate more than 32 kB of memory. The only place where it was necessary to allocate more than 32 kB of memory was the ndpi_init_detection_module() call.

BrainSlayer commented 1 year ago

@vel21ripn yes this might happen on embedded systems with modified kernels which restrict the kmalloc maximum size (which depends on the page size and custom kernel konfiguration). i had the same experience while developing the zstd implementation for zfs in the past. just take care that in case of GFP_ATOMIC nothing is guaranteed. no matter which size. so null checks are very important here. its all about timing. GFP_ATOMIC is restricted to a very short allocation time. if it fails, it returns NULL. also on very small block sizes. GFP_KERNEL however will always return a valid unfragmented pointer of memory is available up to KMALLOC_MAX_SIZE. everything bigger immediatly returns NULL and needs vmalloc fallback then.

BrainSlayer commented 1 year ago

also a hint. i made some optimizations for your current ndpi flow4 tree in dd-wrt. it does compile everything as whole source which all functions declared as static. so like lto for poors in kernel space. this may increase performance and definitly decreases codesize. you may merge my approach, so i dont have to spend too much time in syncing sources over and over again if there are any updates here.

vel21ripn commented 1 year ago

@BrainSlayer Where can I see your build for dd-wrt? The biggest problem with the main project is the static ndpi_packet_struct which precludes the use of multithread. Almost every "git merge" leads to conflicts and their resolution takes time.

BrainSlayer commented 1 year ago

the builds can be downloaded from our website. the source can be found in our svn or at https://github.com/mirror/dd-wrt/tree/master/src/router/ndpi-netfilter

its in sync with your sources almost but contains alot of small changes for compiling all static. nothing really serious. just compare and review it to understand how its made. but i understand if its hard for merging with upstream. i have the same issue everytime i sync your sources with my tree. i do this all manually and it takes some hours every time to get all to work

omikhaylov commented 1 year ago

В ближайшее время сделаю более корректное изменение.

Были ли в недавних коммитах исправления?

vel21ripn commented 1 year ago

Да. Если все будет нормально, то завтра выложу. Там и обновление и исправление ошибок. #1948 IMHO полезный.

vel21ripn commented 1 year ago

Существенный коммит f64efcfee99ee771987f6dda7f0d173fa5c3f565