search

RIOT-OS / Release-Specs

Specification for RIOT releases and corresponding test configurations
4 stars 21 forks source link

Release 2023.04 - RC1 #278

Closed jia200x closed 1 year ago

jia200x commented 1 year ago

This issue lists the status of all tests for the Release Candidate 1 of the 2023.04 release.

Specs tested:

riot-ci commented 1 year ago

Test Report

Task Outcome
✔ 01. Task 03 - Unittests on native separated [tests-ztimer64] PASSED
✔ 03. Task 01 - ICMPv6 multicast echo on native PASSED
✔ 03. Task 02 - ICMPv6 link-local echo on native PASSED
✔ 03. Task 03 - ICMPv6 link-local echo on native (1 hour) PASSED
✔ 03. Task 04 - ICMPv6 stress test on native (1 hour) PASSED
✔ 03. Task 05 - ICMPv6 stress test on native (neighbor cache stress) PASSED
✔ 03. Task 06 - ICMPv6 link-local echo on native (IPv6 fragmentation) PASSED
✔ 04. Task 03 - ICMPv6 echo with large payload PASSED
✔ 04. Task 04 - ICMPv6 echo with iotlab-m3/samr21-xpro 15 minutes PASSED
🟡 04. Task 05 (Experimental) - ICMPv6 multicast echo with samr21-xpro/cc2538 [lille] SKIPPED
🟡 04. Task 06 (Experimental)- ICMPv6 link-local echo with samr21-xpro/cc2538 [lille] SKIPPED
🟡 04. Task 07 (Experimental) - ICMPv6 multicast echo with samr21-xpro/zero + xbee SKIPPED
🟡 04. Task 08 (Experimental) - ICMPv6 echo with samr21-xpro/zero + xbee SKIPPED
✔ 04. Task 09 - ICMPv6 stress test on iotlab-m3 PASSED
✔ 04. Task 10 (Experimental) - ICMPv6 echo with large payload (IPv6 fragmentation) PASSED
✔ 04. Task 11 (Experimental) - ICMPv6 stress test on nrf802154 PASSED
✔ 04. Task 12 (Experimental) - ICMPv6 multicast echo with iotlab-m3/nrf802154 PASSED
✔ 04. Task 13 (Experimental) - ICMPv6 link-local echo with iotlab-m3/nrf802154 PASSED
✔ 05. Task 01 - ICMPv6 echo unicast addresess on native (default route) PASSED
✔ 05. Task 02 - ICMPv6 echo unicast addresess on iotlab-m3 (default route) PASSED
✔ 05. Task 03 - ICMPv6 echo unicast addresess on native (specific route) PASSED
✔ 05. Task 04 - ICMPv6 echo unicast addresess on iotlab-m3 (static route) PASSED
✔ 06. Task 01 - UDP on iotlab-m3 PASSED
✔ 06. Task 02 - UDP on iotlab-m3 (UDP port compression) PASSED
✔ 06. Task 03 - UDP on native (non-existent neighbor) PASSED
✔ 06. Task 04 - UDP on iotlab-m3 (non-existent neighbor) PASSED
✔ 06. Task 05 - Empty UDP on native PASSED
✔ 06. Task 06 - Empty UDP on iotlab-m3 PASSED
✔ 07. Task 01 - ICMPv6 echo on iotlab-m3 with three hops (static route) PASSED
✔ 07. Task 02 - UDP on iotlab-m3 with three hops (static route) PASSED
✔ 07. Task 03 - ICMPv6 echo on iotlab-m3 with three hops (RPL route) PASSED
✔ 07. Task 04 - UDP on iotlab-m3 with three hops (RPL route) PASSED
🟡 07. Task 05 (Experimental) - UDP with large payload on iotlab-m3 with three hops (RPL route) SKIPPED
✔ 08. Task 01 - ICMPv6 echo between native and Linux PASSED
✔ 08. Task 08 - UDP between GNRC and lwIP on iotlab-m3 PASSED
✔ 09. Task 01 - CORD Endpoint PASSED
✔ 09. Task 02 - Confirmable retries [False-0] PASSED
✔ 09. Task 02 - Confirmable retries [True-1] PASSED
✔ 09. Task 03 - Block1 PASSED
✔ 09. Task 04 - Block2 PASSED
✔ 09. Task 05 - Observe registration and notification PASSED
✔ 11. Task 01 - LoRaWAN example PASSED
✔ 11. Task 02 - OTAA join procedure [0] PASSED
✔ 11. Task 02 - OTAA join procedure [3] PASSED
✔ 11. Task 02 - OTAA join procedure [5] PASSED
✔ 11. Task 03 - ABP join procedure [0] PASSED
✔ 11. Task 03 - ABP join procedure [3] PASSED
✔ 11. Task 03 - ABP join procedure [5] PASSED
✔ 11. Task 04 - LoRaWAN device parameters persistence PASSED
✔ 11. Task 05 - GNRC LoRaWAN (Over The Air Activation) [otaa] PASSED
🟡 11. Task 06 - GNRC LoRaWAN (ABP) [abp] SKIPPED
MrKevinWeiss commented 1 year ago

The compile_and_test_for_board.py results in the following issues (will be updated as time goes by):

MrKevinWeiss commented 1 year ago
Task 11 - UDP exchange between iotlab-m3 and Zephyr v2.7.4 -> PASS ``` > udp send fe80::dc63:100:712e:9c3b 4242 "RIOT Testing!" 2023-04-06 13:12:06,637 # udp send fe80::dc63:100:712e:9c3b 4242 "RIOT Testing!" 2023-04-06 13:12:06,639 # Success: sent 13 byte(s) to [fe80::dc63:100:712e:9c3b]:4242 > 2023-04-06 13:12:06,652 # PKTDUMP: data received: 2023-04-06 13:12:06,654 # ~~ SNIP 0 - size: 13 byte, type: NETTYPE_UNDEF (0) 2023-04-06 13:12:06,657 # 00000000 52 49 4F 54 20 54 65 73 74 69 6E 67 21 2023-04-06 13:12:06,659 # ~~ SNIP 1 - size: 8 byte, type: NETTYPE_UDP (4) 2023-04-06 13:12:06,660 # src-port: 4242 dst-port: 4242 2023-04-06 13:12:06,661 # length: 21 cksum: 0xd706 2023-04-06 13:12:06,668 # ~~ SNIP 2 - size: 40 byte, type: NETTYPE_IPV6 (2) 2023-04-06 13:12:06,670 # traffic class: 0x00 (ECN: 0x0, DSCP: 0x00) 2023-04-06 13:12:06,671 # flow label: 0x00000 2023-04-06 13:12:06,673 # length: 21 next header: 17 hop limit: 64 2023-04-06 13:12:06,675 # source address: fe80::dc63:100:712e:9c3b 2023-04-06 13:12:06,678 # destination address: fe80::a41d:cd21:6be0:1774 2023-04-06 13:12:06,680 # ~~ SNIP 3 - size: 24 byte, type: NETTYPE_NETIF (-1) 2023-04-06 13:12:06,681 # if_pid: 6 rssi: -49 lqi: 255 2023-04-06 13:12:06,682 # flags: 0x0 2023-04-06 13:12:06,684 # src_l2addr: DE:63:01:00:71:2E:9C:3B 2023-04-06 13:12:06,685 # dst_l2addr: A6:1D:CD:21:6B:E0:17:74 2023-04-06 13:12:06,687 # ~~ PKT - 4 snips, total size: 85 byte ```
jia200x commented 1 year ago
Task 08.04: CMPv6 echo between iotlab-m3 and Internet host through Linux with 6LowPAN -> PASS ``` 2023-04-06 17:01:40,610 # ping 2a00:1450:4005:801::200e 2023-04-06 17:01:40,744 # 12 bytes from 2a00:1450:4005:801::200e: icmp_seq=0 ttl=117 rssi=-66 dBm time=124.123 ms 2023-04-06 17:01:41,754 # 12 bytes from 2a00:1450:4005:801::200e: icmp_seq=1 ttl=117 rssi=-66 dBm time=123.523 ms 2023-04-06 17:01:42,679 # 12 bytes from 2a00:1450:4005:801::200e: icmp_seq=2 ttl=117 rssi=-66 dBm time=38.835 ms 2023-04-06 17:01:42,680 # 2023-04-06 17:01:42,684 # --- 2a00:1450:4005:801::200e PING statistics --- 2023-04-06 17:01:42,689 # 3 packets transmitted, 3 packets received, 0% packet loss 2023-04-06 17:01:42,694 # round-trip min/avg/max = 38.835/95.493/124.123 ms ```
Task 08.02 ICMPv6 echo between iotlab-m3 and Linux with 6LowPAN -> PASS ``` 2023-04-06 17:02:45,087 # ping 2001:67c:254:b0c0::2 2023-04-06 17:02:45,106 # 12 bytes from 2001:67c:254:b0c0::2: icmp_seq=0 ttl=255 rssi=-66 dBm time=11.209 ms 2023-04-06 17:02:46,119 # 12 bytes from 2001:67c:254:b0c0::2: icmp_seq=1 ttl=255 rssi=-66 dBm time=12.730 ms 2023-04-06 17:02:47,128 # 12 bytes from 2001:67c:254:b0c0::2: icmp_seq=2 ttl=255 rssi=-66 dBm time=11.671 ms 2023-04-06 17:02:47,129 # 2023-04-06 17:02:47,132 # --- 2001:67c:254:b0c0::2 PING statistics --- 2023-04-06 17:02:47,138 # 3 packets transmitted, 3 packets received, 0% packet loss 2023-04-06 17:02:47,142 # round-trip min/avg/max = 11.209/11.870/12.730 ms ```
jia200x commented 1 year ago
01. Task #02 - Unittests on native - PASS ``` main(): This is RIOT! (Version: 2023.07-devel-HEAD) Help: Press s to start test, r to print it is ready READY s START .......................==853316==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! ..............................................................................................................................................................................................................==853316==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0xfff61000; bottom 0x0825e000; size: 0xf7d03000 (-137351168) False positive error reports may follow For details see https://github.com/google/sanitizers/issues/189 ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. OK (1154 tests) ```
jia200x commented 1 year ago
01. Task #04 - Unittests on iotlab-m3 - PASS ``` ssh -t $USER@grenoble.iot-lab.info 'socat - tcp:m3-10.grenoble.iot-lab.info:20000' READY s START .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. OK (1154 tests) ```
MrKevinWeiss commented 1 year ago

It seems none of the 10-icmpv6-error tasks were run automatically. I am finished my vacation and should be able to pick testing back up tomorrow. I will try to run that manually.

jia200x commented 1 year ago

All 10-icmpv6-error test pass. @miri64 triggered them an hour ago: https://github.com/RIOT-OS/RIOT/actions/runs/4687899770/jobs/8307765249

For some reason they were not triggered by this RC1 and consequently not reported back

jia200x commented 1 year ago

I replaced the Contiki test with Contiki-NG (see https://github.com/RIOT-OS/Release-Specs/pull/280)

jia200x commented 1 year ago
04-single-hop-6lowpan-icmp: Task #01 - ICMPv6 link-local echo with iotlab-m3 - PASS ``` 1681481583.268771;m3-14;8 bytes from fe80::fc73:82cc:42b9:b6f9%6: icmp_seq=999 ttl=64 rssi=-46 dBm 1681481583.269018;m3-14; 1681481583.269129;m3-14;--- fe80::fc73:82cc:42b9:b6f9 PING statistics --- 1681481583.284672;m3-14;1000 packets transmitted, 1000 packets received, 0% packet loss ```
MrKevinWeiss commented 1 year ago

04-task02 seems like it passed just scraping failed.

MrKevinWeiss commented 1 year ago

Closing in favor of #281