crossroadsfpga / pigasus

100Gbps Intrusion Detection and Prevention System
663 stars 73 forks source link

Could not get kernel memory #17

Closed lapnd closed 2 years ago

lapnd commented 2 years ago

Hi, Thank for your work. I'm trying to run Pigasus with our Stratix MX board. I'm able to flash FPGA bit stream, the host recognized the FPGA device as

uname -a
Linux mx 4.15.0-180-generic #189-Ubuntu SMP Wed May 18 14:13:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

# output of dmesg

##################################################
# After reboot machine
##################################################
....
[    3.465425] ACPI: If an ACPI driver is available for this device, you should use it instead of the native driver
[    3.465425] lpc_ich: Resource conflict(s) found affecting gpio_ich
[    3.469691] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
[    3.620181] FPGA manager framework
[    3.641212] altera-cvp 0000:01:00.0: Wrong EXT_CAP_ID value 0x0
.....
##################################################
# After insert driver
##################################################
[  227.021978] intel_fpga_pcie_drv: loading out-of-tree module taints kernel.
[  227.022004] intel_fpga_pcie_drv: module verification failed: signature and/or required key missing - tainting kernel
[  227.022360] Mapping BAR: 0
[  227.022378] Mapping BAR: 1
[  227.022379] Mapping BAR: 2
[  227.022386] Mapping BAR: 3
[  227.022387] Mapping BAR: 4
[  227.022387] Mapping BAR: 5

Howerver, when running Pigagus, I got Could not get kernel memory message

pigasus -c snort.lua --patterns ./rule_list
--------------------------------------------------
o")~   Snort++ 3.0.0-249
--------------------------------------------------
Loading snort.lua:
        ssh
        pop
        reject
        stream_tcp
        dce_http_proxy
        normalizer
        stream_udp
        search_engine
        dce_smb
        ips
        binder
        detection
        modbus
        network
        sip
        ssl
        dce_http_server
        dce_tcp
        smtp
        ftp_data
        ftp_server
        telnet
        rpc_decode
        http_inspect
        classifications
        stream
        stream_ip
        process
        profiler
        event_queue
        dnp3
        active
        ftp_client
        references
        dns
        dce_udp
        imap
Finished snort.lua.
Loading rules:
Loading ../rules/sample.rules:
Finished ../rules/sample.rules.
Finished rules.
--------------------------------------------------
rule counts
       total rules loaded: 1
               text rules: 1
            option chains: 1
            chain headers: 1
--------------------------------------------------
port rule counts
             tcp     udp    icmp      ip
     any       1       0       0       0
   total       1       0       0       0
Creating Port Groups....

IP-SRC 0 Port Groups in Port Table

IP-DST 0 Port Groups in Port Table

IP-ANY  PortObject any  Id:0  Ports:1 Rules:0
 {
 Ports [any ]
 }
PortGroup rule summary (ports):

ICMP-SRC 0 Port Groups in Port Table

ICMP-DST 0 Port Groups in Port Table

ICMP-ANY  PortObject any  Id:0  Ports:1 Rules:0
 {
 Ports [any ]
 }
PortGroup rule summary (ports):

TCP-SRC 0 Port Groups in Port Table

TCP-DST 0 Port Groups in Port Table

TCP-ANY  PortObject any  Id:0  Ports:1 Rules:1
 {
 Ports [any ]
 }
init mpse: hyperscan
FP port 1:60200:1 packet[6] = '....".' |FA A5 F8 FB 22 88 | ( user )
PortGroup rule summary (ports):
        packet: 1

UDP-SRC 0 Port Groups in Port Table

UDP-DST 0 Port Groups in Port Table

UDP-ANY  PortObject any  Id:0  Ports:1 Rules:0
 {
 Ports [any ]
 }
PortGroup rule summary (ports):

SVC-ANY  PortObject any  Id:0  Ports:1 Rules:0
 {
 Ports [any ]
 }
PortGroup rule summary (ports):
Port Groups Done....
Creating Rule Maps....
Rule Maps Done....
Creating Service Based Rule Maps....
+--------------------------------
| Service-PortGroup Table Summary 
---------------------------------
---------------------------------
Service Based Rule Maps Done....
--------------------------------------------------
fast pattern port groups        src     dst     any
                   packet:        0       0       1
--------------------------------------------------
search engine
                instances: 1
                 patterns: 1
init mpse: hyperscan
init mpse: hyperscan
init mpse: hyperscan
init mpse: hyperscan
init mpse: hyperscan
init mpse: hyperscan
init mpse: hyperscan
--------------------------------------------------
pcap DAQ configured to passive.
Commencing packet processing
elapsed = 3400402671 
clock scale = 3400 
usec 1000118.437500 
Core_id:0
Could not get kernel memory!
nb matches: 0
nb no matches: 0
--------------------------------------------------
Packet Statistics
--------------------------------------------------
Module Statistics
--------------------------------------------------
Summary Statistics
--------------------------------------------------
timing
                  runtime: 00:00:02
                  seconds: 2.987
                  packets: 0
                 pkts/sec: 0
o")~   Snort exiting

I added few printk messages and found that the address is 0

# dmesg
[ 2141.817004] set_kmem_size
[ 2141.817005] dev_bk->kmem_info.size=0
[ 2141.817006] dev_bk->kmem_info.virt_addr=0
[ 2141.817006] dev_bk->kmem_info.bus_addr=0
[ 2141.817091] dev_bk->kmem_info.virt_addr=0
[ 2141.817092] could not allocate 2
[ 2141.817092] Core_id:0

Could you please give few hints on what would be wrong/missing steps?

Thank you!

lapnd commented 2 years ago

It seems due to lack of memory. I have only 16GB RAM only. After adding more 8GB, the error could not allocate 2 gone away but new error comes: Could not get mmap kernel memory!

usec 1000087.875000 
Core_id:0
Could not get mmap kernel memory!
nb matches: 0
nb no matches: 0

I wonder if it's still lack of memory issue. Can you please tell me how much memory is needed to run Pigasus? Thank you!

hsadok commented 2 years ago

Hi, thank you for trying Pigasus and for reporting this issue!

This error is actually not due to insufficient memory. The issue is that Pigasus requires CMA to be enabled in the kernel.

The kernel must be compiled with support for CMA and you must preallocate a CMA memory region at boot time.

To allocate a CMA region at boot time, add the following to your /etc/default/grub configuration file.

GRUB_CMDLINE_LINUX="cma=1G"

If the option GRUB_CMDLINE_LINUX already exists, append cma=1G to it, making sure there is a space between the existing options and the new ones. For example:

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash cma=1GB"

Then run:

sudo update-grub

And reboot the server:

sudo reboot

Important. If your kernel is compiled with CMA disabled, you need to recompile the kernel enabling CMA. I found that this answer on StackOverflow gives good instructions on how to do it: https://stackoverflow.com/a/60231594/2027390

Thank you for reporting this. This reminds me that we should add instructions about this to the README, or start using hugepages to avoid the need to recompile the kernel.

lapnd commented 2 years ago

Hi @hsadok Thank your for your helps. I'm able to start Pigagus now.

--------------------------------------------------
o")~   Snort++ 3.0.0-249
--------------------------------------------------
Loading snort.lua:
        ssh
        pop
        reject
        stream_tcp
        dce_http_proxy
        normalizer
        stream_udp
        search_engine
        dce_smb
        ips
        binder
        detection
        modbus
        network
        sip
        ssl
        dce_http_server
        dce_tcp
        smtp
        ftp_data
        ftp_server
        telnet
        rpc_decode
        http_inspect
        classifications
        stream
        stream_ip
        process
        profiler
        event_queue
        dnp3
        active
        ftp_client
        references
        dns
        dce_udp
        imap
Finished snort.lua.
Loading rules:
Loading ../rules/sample.rules:
Finished ../rules/sample.rules.
Finished rules.
--------------------------------------------------
rule counts
       total rules loaded: 1
               text rules: 1
            option chains: 1
            chain headers: 1
--------------------------------------------------
port rule counts
             tcp     udp    icmp      ip
     any       1       0       0       0
   total       1       0       0       0
Creating Port Groups....

IP-SRC 0 Port Groups in Port Table

IP-DST 0 Port Groups in Port Table

IP-ANY  PortObject any  Id:0  Ports:1 Rules:0
 {
 Ports [any ]
 }
PortGroup rule summary (ports):

ICMP-SRC 0 Port Groups in Port Table

ICMP-DST 0 Port Groups in Port Table

ICMP-ANY  PortObject any  Id:0  Ports:1 Rules:0
 {
 Ports [any ]
 }
PortGroup rule summary (ports):

TCP-SRC 0 Port Groups in Port Table

TCP-DST 0 Port Groups in Port Table

TCP-ANY  PortObject any  Id:0  Ports:1 Rules:1
 {
 Ports [any ]
 }
init mpse: hyperscan
FP port 1:60200:1 packet[6] = '....".' |FA A5 F8 FB 22 88 | ( user )
PortGroup rule summary (ports):
        packet: 1

UDP-SRC 0 Port Groups in Port Table

UDP-DST 0 Port Groups in Port Table

UDP-ANY  PortObject any  Id:0  Ports:1 Rules:0
 {
 Ports [any ]
 }
PortGroup rule summary (ports):

SVC-ANY  PortObject any  Id:0  Ports:1 Rules:0
 {
 Ports [any ]
 }
PortGroup rule summary (ports):
Port Groups Done....
Creating Rule Maps....
Rule Maps Done....
Creating Service Based Rule Maps....
+--------------------------------
| Service-PortGroup Table Summary 
---------------------------------
---------------------------------
Service Based Rule Maps Done....
--------------------------------------------------
fast pattern port groups        src     dst     any
                   packet:        0       0       1
--------------------------------------------------
search engine
                instances: 1
                 patterns: 1
init mpse: hyperscan
init mpse: hyperscan
init mpse: hyperscan
init mpse: hyperscan
init mpse: hyperscan
init mpse: hyperscan
init mpse: hyperscan
--------------------------------------------------
pcap DAQ configured to passive.
Commencing packet processing
elapsed = 3399618373 
clock scale = 3399 
usec 1000181.937500 
Core_id:0
fpga_reg[0] = 0x00000000 
fpga_reg[1] = 0x00000000 
fpga_reg[2] = 0x70200000 
fpga_reg[3] = 0x00000005 
fpga_reg[4] = 0x00000000 
fpga_reg[5] = 0x00000000 
fpga_reg[6] = 0x00000000 
fpga_reg[7] = 0x00000000 
fpga_reg[8] = 0x00000000 
fpga_reg[9] = 0x00000000 
fpga_reg[10] = 0x00000000 
fpga_reg[11] = 0x00000000 
fpga_reg[12] = 0x00000000 
fpga_reg[13] = 0x00000000 
fpga_reg[14] = 0x00000000 
fpga_reg[15] = 0x00000000 
fpga_reg[16] = 0x00000000 
fpga_reg[17] = 0x00000000 
fpga_reg[18] = 0x70200000 
fpga_reg[19] = 0x00000005 
fpga_reg[20] = 0x00000000 
fpga_reg[21] = 0x00000000 
fpga_reg[22] = 0x00000000 
fpga_reg[23] = 0x00000000 
fpga_reg[24] = 0x00000000 
fpga_reg[25] = 0x00000000 
fpga_reg[26] = 0x00000000 
fpga_reg[27] = 0x00000000 
fpga_reg[28] = 0x00000000 
fpga_reg[29] = 0x00000000 
fpga_reg[30] = 0x00000000 
fpga_reg[31] = 0x00000000 
hsadok commented 2 years ago

Awesome!