A large number of LTP test cases fails due to commit 8f57715f8ef6

[coklm]

When I run latest ltp testcase testcases/network/stress/udp/uni-basic/udp6-uni-basic07 with latest kernel, it failed in “ltpapicmd.c:188: Failed to configure SAD/SPD on the local host”. I found that the reason for failure was this command: output_ipsec_conf src \ $IPSEC_PROTO $IPSEC_MODE $SPI $lhost_addr $rhost_addr \ | setkey -c 2>&1 | tee $ipsec_log

This command failed when use setkey -c to add this failed: spdadd $src_ipaddr $dst_ipaddr any 　　-P out ipsec $protocol/tunnel/${src_ipaddr}-${dst_ipaddr}/use ;

It returns Invalid argument.

I found this failed due to commit 8f57715f8ef6(af_key: Reject optional tunnel/BEET mode templates in outbound policies),should ltp testcase update?

[metan-ucw]

The actual commit hash is cf3128a7aca5. I suppose that we should drop the line since kernel started rejecting outbound policies, does the tests work with that particular line removed?

[coklm]

Do you mean remove this line is in output_ipsec_conf scripts? Many testcases use output_ipsec_conf scripts，i change 'use' to 'require' and some testcases passed. But some testcases still failed, these testcases used ipcomp/tunnel/require ipsec policy failed in 'ping -I $src_ifname -c 1 $dst_ipv4addr -w 1' command, for example testcase udp4-uni-basic07. Should ltp testcases can adapt latest kernel?

[pevik]

I agree that setkey should not be used, it's deprecated by ip xfrm and that's why distros stopped using it (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=917847).

Use of deprecated tool is usually a sign of poor quality of the test and these network stress tests (files under testcases/network/stress/{icmp,tcp,udp}/) are no exception - whatever uses anything from testcases/network/stress/ns-tools/ instead of testcases/lib/tst_net.sh is one of the worst parts of LTP (it does not use even the legacy API) and should be either converted to new shell API + use testcases/lib/tst_net.sh or deleted.

I have mentioned that, when I update https://github.com/linux-test-project/ltp/issues/128 description. It looks to me that tests are duplicate to the tests, which are using ipsec_lib.sh and are testing either with ping via tst_ping() (they work well, use new API). Unlike other old tests which got a runtest file back in 241ec180c31cb45408cfdd35a9ed9acabde811b4, these were added a year before and does not even have a runtest file.

They are unusable in this form I hesitated to delete them before fully exploring what they do. From a quick look some of them e.g. do network delay via tc qdisc ... delay (see man tc-netem(8)). We use it only in tcp_fastopen_run.sh. But I don't know how many tests we can add to runtest/net_stress.ipsec_{dccp,icmp,sctp,tcp,udp} because they are already big enough.