tempesta-tech / tempesta

All-in-one solution for high performance web content delivery and advanced protection against DDoS and web attacks
https://tempesta-tech.com/
GNU General Public License v2.0
621 stars 103 forks source link

on-the-fly-configuration: `listen` directive does not work correctly #2017

Closed RomanBelozerov closed 9 months ago

RomanBelozerov commented 11 months ago

Related to #2043 and probably should be fixed at the same time

Scope

Tempesta - 4e6c0342b5ee113753ffc61b8a89a784dbe463f5

Tests to reproduce: reconf.test_reconf_base.TestListenReconfHttp.test_reconf_ip_ipv4_to_ipv4_port_443 reconf.test_reconf_base.TestListenReconfHttp.test_reconf_ip_ipv4_to_ipv4_port_default

Trace
[  221.035218] BUG: kernel NULL pointer dereference, address: 0000000000000004
[  221.035739] #PF: supervisor write access in kernel mode
[  221.036122] #PF: error_code(0x0002) - not-present page
[  221.036497] PGD 0 P4D 0 
[  221.036688] Oops: 0002 [#1] SMP NOPTI
[  221.036967] CPU: 3 PID: 4677 Comm: python3 Tainted: G           OE     5.10.35.tfw-ba90726 #1
[  221.037579] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.15.0-1 04/01/2014
[  221.038165] RIP: 0010:_raw_spin_lock+0xc/0x30
[  221.038480] Code: 0f c1 07 a9 ff 01 00 00 75 07 4c 89 e0 41 5c 5d c3 e8 d8 47 57 ff 4c 89 e0 41 5c 5d c3 90 0f 1f 44 00 00 31 c0 ba 01 00 00 00  0f b1 17 75 01 c3 55 89 c6 48 89 e5 e8 92 30 57 ff 66 90 5d c3
[  221.039802] RSP: 0018:ffffbe7e40150738 EFLAGS: 00010246
[  221.040177] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000080140013
[  221.040696] RDX: 0000000000000001 RSI: 00000000fffffe01 RDI: 0000000000000004
[  221.041216] RBP: ffffbe7e40150760 R08: 0000000000000001 R09: ffffffffc0b38c01
[  221.041731] R10: ffff9c87296db180 R11: 0000000000000001 R12: ffff9c8637d58000
[  221.042246] R13: 0000000000000000 R14: 0000000000000004 R15: ffffbe7e401507c8
[  221.042755] FS:  00007f195ad94640(0000) GS:ffff9c87f0ec0000(0000) knlGS:0000000000000000
[  221.043340] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  221.043757] CR2: 0000000000000004 CR3: 0000000129752004 CR4: 0000000000770ee0
[  221.044269] PKRU: 55555554
[  221.044884] Call Trace:
[  221.045450]  
[  221.045995]  ? frang_tls_handler+0x3b/0xc0 [tempesta_fw]
[  221.046763]  tfw_tls_over+0x16/0x20 [tempesta_fw]
[  221.047472]  ttls_handshake_server_step+0x249/0x1fb0 [tempesta_tls]
[  221.048293]  ? netif_rx+0x1b/0x90
[  221.048916]  ? __alloc_pages_nodemask+0x193/0x330
[  221.049614]  ? bzero_fast+0xe/0x10 [tempesta_lib]
[  221.050311]  ttls_recv+0x349/0x850 [tempesta_tls]
[  221.051008]  ss_skb_process+0xf5/0x140 [tempesta_fw]
[  221.051713]  ? ttls_handle_alert+0x40/0x40 [tempesta_tls]
[  221.052458]  tfw_tls_connection_recv+0xad/0x3a0 [tempesta_fw]
[  221.053244]  ss_tcp_process_data+0x1ec/0x3f0 [tempesta_fw]
[  221.053992]  ? __enqueue_entity+0x96/0xa0
[  221.054640]  ss_tcp_data_ready+0x44/0xe0 [tempesta_fw]
[  221.055359]  tcp_data_ready+0x2b/0xd0
[  221.055952]  tcp_data_queue+0x452/0xd10
[  221.056542]  tcp_rcv_established+0x230/0x670
[  221.057167]  ? sk_filter_trim_cap+0xde/0x240
[  221.057799]  tcp_v4_do_rcv+0x140/0x200
[  221.058392]  tcp_v4_rcv+0xcfd/0xe10
[  221.058956]  ip_protocol_deliver_rcu+0x30/0x1b0
[  221.059601]  ip_local_deliver_finish+0x48/0x60
[  221.060231]  ip_local_deliver+0xfa/0x110
[  221.060834]  ? ip_protocol_deliver_rcu+0x1b0/0x1b0
[  221.061505]  ip_rcv_finish+0x87/0xa0
[  221.062070]  ip_rcv+0xcc/0xe0
[  221.062592]  ? ip_rcv_finish_core.isra.0+0x420/0x420
[  221.063252]  __netif_receive_skb_one_core+0x88/0xa0
[  221.063892]  __netif_receive_skb+0x18/0x60
[  221.064477]  process_backlog+0xa9/0x160
[  221.065081]  net_rx_action+0x13e/0x390
[  221.065694]  ? vring_interrupt+0x5f/0xc0
[  221.066284]  __do_softirq+0xd9/0x291
[  221.066839]  asm_call_irq_on_stack+0xf/0x20
[  221.067421]  
[  221.067845]  do_softirq_own_stack+0x3d/0x50
[  221.068402]  do_softirq.part.0+0x46/0x50
[  221.068948]  __local_bh_enable_ip+0x50/0x60
[  221.069489]  ip_finish_output2+0x1ab/0x590
[  221.070030]  ? __cgroup_bpf_run_filter_skb+0x3c3/0x3d0
[  221.070635]  __ip_finish_output+0xd8/0x220
[  221.071181]  ip_finish_output+0x2d/0xb0
[  221.071714]  ip_output+0x7a/0x100
[  221.072203]  ? __ip_finish_output+0x220/0x220
[  221.072759]  ip_local_out+0x3d/0x50
[  221.073248]  __ip_queue_xmit+0x17a/0x470
[  221.073764]  ? __ext4_journal_get_write_access+0x7a/0x120
[  221.074387]  ip_queue_xmit+0x15/0x20
[  221.074895]  __tcp_transmit_skb+0xa20/0xbe0
[  221.075437]  tcp_write_xmit+0x2cc/0x11b0
[  221.075957]  __tcp_push_pending_frames+0x37/0x100
[  221.076529]  tcp_push+0xfc/0x100
[  221.077002]  tcp_sendmsg_locked+0xd36/0xe70
[  221.077535]  tcp_sendmsg+0x2d/0x50
[  221.078039]  inet_sendmsg+0x43/0x70
[  221.078522]  sock_sendmsg+0x5e/0x70
[  221.079005]  sock_write_iter+0x93/0xf0
[  221.079498]  new_sync_write+0x192/0x1b0
[  221.079995]  vfs_write+0x185/0x250
[  221.080457]  ksys_write+0xb1/0xe0
[  221.080911]  __x64_sys_write+0x1a/0x20
[  221.081381]  do_syscall_64+0x38/0x90
[  221.081846]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[  221.082418] RIP: 0033:0x7f195ff66aaf
[  221.082888] Code: 89 54 24 18 48 89 74 24 10 89 7c 24 08 e8 59 bf f7 ff 48 8b 54 24 18 48 8b 74 24 10 41 89 c0 8b 7c 24 08 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 31 44 89 c7 48 89 44 24 08 e8 9c bf f7 ff 48
[  221.084662] RSP: 002b:00007f195ad92930 EFLAGS: 00000293 ORIG_RAX: 0000000000000001
[  221.085446] RAX: ffffffffffffffda RBX: 000000000000007e RCX: 00007f195ff66aaf
[  221.086200] RDX: 000000000000007e RSI: 00007f1954012d10 RDI: 0000000000000027
[  221.086957] RBP: 00007f1954004390 R08: 0000000000000000 R09: 00007f195ed69b60
[  221.087709] R10: 0000000000000000 R11: 0000000000000293 R12: 00007f1954012d10
[  221.088458] R13: 000000000000007e R14: 00007f195ad92a00 R15: 00007f19540021f0
[  221.089221] Modules linked in: tempesta_fw(OE) tempesta_db(OE) sha256_ssse3 sha512_ssse3 tempesta_tls(OE) tempesta_lib(OE) tls xt_conntrack nft_chain_nat xt_MASQUERADE nf_nat nf_conntrack_netlink nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 xfrm_user xfrm_algo nft_counter xt_addrtype nft_compat nf_tables libcrc32c nfnetlink br_netfilter bridge stp llc overlay snd_hda_codec_generic ledtrig_audio snd_hda_intel snd_intel_dspcfg snd_hda_codec snd_hda_core snd_hwdep binfmt_misc snd_pcm snd_seq_midi snd_seq_midi_event snd_rawmidi intel_rapl_msr intel_rapl_common kvm_intel kvm snd_seq nls_iso8859_1 snd_seq_device crct10dif_pclmul ghash_clmulni_intel snd_timer aesni_intel crypto_simd cryptd glue_helper snd soundcore joydev qxl input_leds drm_ttm_helper ttm serio_raw drm_kms_helper cec fb_sys_fops syscopyarea sysfillrect sysimgblt mac_hid qemu_fw_cfg sch_fq_codel msr parport_pc ppdev lp ramoops parport drm reed_solomon efi_pstore ip_tables x_tables autofs4 hid_generic usbhid hid crc32_pclmul
[  221.089261]  virtio_net ahci lpc_ich psmouse libahci i2c_i801 i2c_smbus net_failover virtio_rng failover virtio_blk
[  221.098793] CR2: 0000000000000004
[  221.099384] ---[ end trace bbbd84a74f0386f5 ]---
[  221.100062] RIP: 0010:_raw_spin_lock+0xc/0x30
[  221.100737] Code: 0f c1 07 a9 ff 01 00 00 75 07 4c 89 e0 41 5c 5d c3 e8 d8 47 57 ff 4c 89 e0 41 5c 5d c3 90 0f 1f 44 00 00 31 c0 ba 01 00 00 00  0f b1 17 75 01 c3 55 89 c6 48 89 e5 e8 92 30 57 ff 66 90 5d c3
[  221.102796] RSP: 0018:ffffbe7e40150738 EFLAGS: 00010246
[  221.103553] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000080140013
[  221.104431] RDX: 0000000000000001 RSI: 00000000fffffe01 RDI: 0000000000000004
[  221.105316] RBP: ffffbe7e40150760 R08: 0000000000000001 R09: ffffffffc0b38c01
[  221.106210] R10: ffff9c87296db180 R11: 0000000000000001 R12: ffff9c8637d58000
[  221.107102] R13: 0000000000000000 R14: 0000000000000004 R15: ffffbe7e401507c8
[  221.107993] FS:  00007f195ad94640(0000) GS:ffff9c87f0ec0000(0000) knlGS:0000000000000000
[  221.108961] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  221.109781] CR2: 0000000000000004 CR3: 0000000129752004 CR4: 0000000000770ee0
[  221.110680] PKRU: 55555554
[  221.111278] Kernel panic - not syncing: Fatal exception in interrupt
[  221.112370] Kernel Offset: 0x27a00000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
[  221.113588] Rebooting in 1 seconds..
[  222.106344] ACPI MEMORY or I/O RESET_REG.

Tests to reproduce: reconf.test_reconf_base.TestListenReconfHttps.test_reconf_ip_ipv4_to_ipv4_port_443 reconf.test_reconf_base.TestListenReconfHttps.test_reconf_ip_ipv4_to_ipv4_port_default reconf.test_reconf_base.TestListenReconfH2.test_reconf_ip_ipv4_to_ipv4_port_443 reconf.test_reconf_base.TestListenReconfH2.test_reconf_ip_ipv4_to_ipv4_port_default

[tempesta fw] Warning: Parser error: state=Req_MethodUnknown input(-0)=0x16('\x16\x03\x01\x02') data_len=517 off=0

Tests to reproduce: reconf.test_reconf_base.TestListenReconfHttp.test_reconf_proto_h2 reconf.test_reconf_base.TestListenReconfHttp.test_reconf_proto_https

[ 8021.544550] [tempesta tls] Warning: bad TLS version 69:84
[ 8021.544554] [tempesta tls] Warning: [::ffff:127.0.0.2] Bad TLS record (err -0xFFFFF003)

Tests to reproduce: reconf.test_reconf_base.TestListenReconfH2.test_reconf_proto_http reconf.test_reconf_base.TestListenReconfHttps.test_reconf_proto_http

[tempesta tls] Warning: [::ffff:127.0.0.2] ClientHello: cannot find matching ALPN for h2

Tests to reproduce: reconf.test_reconf_base.TestListenReconfHttps.test_reconf_proto_h2

Tests to reproduce: reconf.test_reconf_base.TestListenReconfH2.test_reconf_proto_https

EvgeniiMekhanik commented 9 months ago

Fixed by 2045