tempesta-tech / tempesta-test

Test suite for Tempesta FW
11 stars 4 forks source link

Send HTTP response in small chunks and forward it to an h2 client #431

Closed krizhanovsky closed 1 year ago

krizhanovsky commented 1 year ago

Need to implement the test for responses discussed in https://github.com/tempesta-tech/tempesta/pull/1820#discussion_r1126259918

RomanBelozerov commented 1 year ago

Tempesta warning for test:


[18926.161414] ------------[ cut here ]------------
[18926.162260] WARNING: CPU: 2 PID: 67707 at /root/tempesta/fw/tls.c:455 tfw_tls_encrypt+0x71b/0x750 [tempesta_fw]
[18926.164263] Modules linked in: tempesta_fw(OE) tempesta_db(OE) tempesta_tls(OE) tempesta_lib(OE) tcp_diag(E) inet_diag(E) xt_nat(E) xt_tcpudp(E) veth(E) sha256_ssse3(E) sha512_ssse3(E) sha512_generic(E) nfnetlink_queue(E) nfnetlink_log(E) bluetooth(E) jitterentropy_rng(E) drbg(E) ansi_cprng(E) ecdh_generic(E) ecc(E) cfg80211(E) uinput(E) xt_conntrack(E) nft_chain_nat(E) xt_MASQUERADE(E) nf_nat(E) nf_conntrack_netlink(E) nf_conntrack(E) nf_defrag_ipv6(E) nf_defrag_ipv4(E) xfrm_user(E) xfrm_algo(E) nft_counter(E) xt_addrtype(E) nft_compat(E) nf_tables(E) libcrc32c(E) nfnetlink(E) br_netfilter(E) bridge(E) stp(E) llc(E) overlay(E) hid_generic(E) usbhid(E) hid(E) rfkill(E) intel_rapl_msr(E) intel_rapl_common(E) intel_pmc_core(E) crc32_pclmul(E) ghash_clmulni_intel(E) sr_mod(E) snd_ens1371(E) aesni_intel(E) libaes(E) cdrom(E) snd_ac97_codec(E) crypto_simd(E) cryptd(E) ata_generic(E) glue_helper(E) ac97_bus(E) gameport(E) uhci_hcd(E) snd_rawmidi(E) vmw_balloon(E) snd_seq_device(E) ehci_pci(E)
[18926.164309]  ata_piix(E) snd_pcm(E) ehci_hcd(E) snd_timer(E) libata(E) joydev(E) snd(E) pcspkr(E) usbcore(E) e1000(E) sg(E) soundcore(E) i2c_piix4(E) vsock_loopback(E) vmw_vsock_virtio_transport_common(E) ac(E) button(E) vmw_vsock_vmci_transport(E) vsock(E) vmw_vmci(E) msr(E) parport_pc(E) ppdev(E) lp(E) parport(E) fuse(E) configfs(E) ip_tables(E) x_tables(E) autofs4(E) ext4(E) crc32c_generic(E) crc16(E) mbcache(E) jbd2(E) sd_mod(E) t10_pi(E) crc_t10dif(E) crct10dif_generic(E) vmwgfx(E) drm_kms_helper(E) cec(E) ttm(E) drm(E) mptspi(E) crct10dif_pclmul(E) crct10dif_common(E) scsi_transport_spi(E) psmouse(E) mptscsih(E) evdev(E) mptbase(E) crc32c_intel(E) scsi_mod(E) serio_raw(E) [last unloaded: tempesta_lib]
[18926.190279] CPU: 2 PID: 67707 Comm: python3 Tainted: G           OE     5.10.35+ #10
[18926.191674] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
[18926.193635] RIP: 0010:tfw_tls_encrypt+0x71b/0x750 [tempesta_fw]
[18926.194721] Code: c1 e4 05 49 01 fc 48 c1 e0 05 4c 01 e0 4c 89 64 24 50 48 89 44 24 20 48 89 44 24 38 e9 9f fb ff ff 48 8b 45 28 e9 18 fd ff ff <0f> 0b e9 87 fd ff ff 0f 0b e9 35 ff ff ff 0f 0b e9 25 ff ff ff 0f
[18926.198206] RSP: 0018:ffffa736404ecb38 EFLAGS: 00010297
[18926.199576] RAX: 0000000000000068 RBX: 000000000000014e RCX: 0000000000000010
[18926.201366] RDX: ffffcc74c91ea3c2 RSI: 0000000000000020 RDI: ffff9a738293c000
[18926.202625] RBP: ffff9a7338785580 R08: 0000000000000000 R09: 0000000000000001
[18926.203855] R10: ffff9a7447a8f000 R11: 0000000000000400 R12: 0000000000000067
[18926.205221] R13: ffff9a73b26053c0 R14: ffff9a738293c000 R15: 0000000000000067
[18926.206485] FS:  00007f551c818700(0000) GS:ffff9a745a080000(0000) knlGS:0000000000000000
[18926.207889] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[18926.209031] CR2: 00007f551dc00b70 CR3: 00000001aef36003 CR4: 0000000000370ee0
[18926.210281] Call Trace:
[18926.211280]  <IRQ>
[18926.212892]  ? __dev_queue_xmit+0x376/0x8b0
[18926.214217]  ? nf_confirm+0xcb/0xf0 [nf_conntrack]
[18926.215276]  ? ip_finish_output2+0x1ab/0x590
[18926.216056]  ? ipv4_link_failure+0x1b0/0x1b0
[18926.216957]  ? __ip_queue_xmit+0x188/0x410
[18926.217704]  ? tfw_http_msg_process_generic+0x50e/0x5e0 [tempesta_fw]
[18926.218840]  ? __tcp_transmit_skb+0xa0e/0xbc0
[18926.219821]  ? skb_copy_bits+0x1da/0x210
[18926.220576]  ? tdb_htrie_lookup+0x54/0x80 [tempesta_db]
[18926.222316]  ? __cgroup_bpf_run_filter_skb+0x11b/0x1e0
[18926.223441]  ? __frang_init_acc+0x10/0x10 [tempesta_fw]
[18926.224813]  ? tempesta_sock_tcp_rcv+0x31/0x40
[18926.225913]  ? security_sock_rcv_skb+0x2a/0x40
[18926.226716]  ? tcp_add_backlog+0x30e/0x440
[18926.227444]  tfw_sc_write_xmit+0xfe/0x2c0 [tempesta_fw]
[18926.228358]  tcp_write_xmit+0x3cb/0x1280
[18926.229138]  __tcp_push_pending_frames+0x32/0xf0
[18926.229964]  ss_tx_action+0x2d7/0x620 [tempesta_fw]
[18926.230821]  net_tx_action+0x94/0x240
[18926.231686]  __do_softirq+0xcf/0x284
[18926.232678]  asm_call_irq_on_stack+0x12/0x20
[18926.233526]  </IRQ>
[18926.234155]  do_softirq_own_stack+0x37/0x40
[18926.235271]  do_softirq+0x5e/0x70
[18926.235873]  __local_bh_enable_ip+0x4b/0x50
[18926.236614]  ip_finish_output2+0x1ab/0x590
[18926.237444]  __ip_queue_xmit+0x180/0x410
[18926.238362]  ? lock_timer_base+0x61/0x80
[18926.239074]  __tcp_transmit_skb+0xa0e/0xbc0
[18926.239829]  tcp_write_xmit+0x401/0x1280
[18926.240526]  __tcp_push_pending_frames+0x32/0xf0
[18926.241437]  tcp_sendmsg_locked+0xa32/0xb50
[18926.242245]  tcp_sendmsg+0x28/0x40
[18926.242898]  sock_sendmsg+0x57/0x60
[18926.243509]  __sys_sendto+0xee/0x150
[18926.244330]  ? do_user_addr_fault+0x1cf/0x3f0
[18926.245105]  ? switch_fpu_return+0x40/0xb0
[18926.245935]  __x64_sys_sendto+0x25/0x30
[18926.246624]  do_syscall_64+0x33/0x80
[18926.247259]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[18926.248128] RIP: 0033:0x7f5521392494
[18926.248764] Code: 89 4c 24 1c e8 8d f8 ff ff 44 8b 54 24 1c 8b 3c 24 45 31 c9 89 c5 48 8b 54 24 10 48 8b 74 24 08 45 31 c0 b8 2c 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 34 89 ef 48 89 04 24 e8 b9 f8 ff ff 48 8b 04
[18926.252103] RSP: 002b:00007f551c816ce0 EFLAGS: 00000246 ORIG_RAX: 000000000000002c
[18926.253565] RAX: ffffffffffffffda RBX: 00007f551c816db0 RCX: 00007f5521392494
[18926.254787] RDX: 0000000000000001 RSI: 00007f5520a7eb00 RDI: 0000000000000024
[18926.256002] RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000
[18926.257368] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
[18926.258609] R13: 0000000000000000 R14: 00007f551c816db0 R15: 0000000000000001
[18926.260135] ---[ end trace bf3cdbaf4ed929d5 ]---
krizhanovsky commented 1 year ago

As discussed, we need a good coverage for heavily chunked test cases. If Tempesta crashes, then please create new tasks. See https://github.com/tempesta-tech/tempesta-test/pull/195 and #115.