search

RIOT-OS / Release-Specs

Specification for RIOT releases and corresponding test configurations
4 stars 21 forks source link

Release 2022.07 - RC2 #254

Closed chrysn closed 2 years ago

chrysn commented 2 years ago

This issue lists the status of all tests for the Release Candidate 2 of the 2022.07 release.

Specs tested:

riot-ci commented 2 years ago

Test Report

Task Outcome
✔ 01. Task 02 - Unittests on native PASSED
✔ 01. Task 03 - Unittests on native separated [tests-analog_util] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-base64] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-bcd] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-bitfield] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-bloom] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-bluetil] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-checksum] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-chunked_ringbuffer] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-clif] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-color] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-core] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-credman] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-div] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-dns_cache] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-ecc] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-fib] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-fib_sr] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-fmt] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-frac] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-gcoap] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-gnrc_ipv6] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-gnrc_ipv6_hdr] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-gnrc_ipv6_nib] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-gnrc_mac_internal] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-gnrc_netif_pktq] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-gnrc_sixlowpan_frag_vrb] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-gnrc_udp] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-hashes] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-ieee802154] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-inet_csum] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-ipv4_addr] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-ipv6_addr] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-ipv6_hdr] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-kernel_defines] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-luid] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-matstat] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-mtd] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-nanocoap_cache] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-netopt] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-netreg] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-phydat] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-pkt] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-pktbuf] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-pktqueue] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-printf_float] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-priority_pktqueue] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-rtc] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-rtt_rtc] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-saul_reg] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-scanf_float] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-seq] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-sht1x] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-sixlowpan] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-sixlowpan_ctx] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-sixlowpan_sfr] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-sock_util] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-timex] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-tsrb] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-turo] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-uri_parser] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-ut_process] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-uuid] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-vfs] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-zptr] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-ztimer64] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-ztimer] PASSED
✔ 01. Task 04 - Unittests on iotlab-m3 PASSED
✔ 03. Task 01 - ICMPv6 multicast echo on native PASSED
✔ 03. Task 02 - ICMPv6 link-local echo on native PASSED
✔ 03. Task 03 - ICMPv6 link-local echo on native (1 hour) PASSED
✔ 03. Task 04 - ICMPv6 stress test on native (1 hour) PASSED
✔ 03. Task 05 - ICMPv6 stress test on native (neighbor cache stress) PASSED
✔ 03. Task 06 - ICMPv6 link-local echo on native (IPv6 fragmentation) PASSED
✔ 04. Task 01 - ICMPv6 link-local echo with iotlab-m3 PASSED
✔ 04. Task 02 - ICMPv6 multicast echo with iotlab-m3/samr21-xpro PASSED
✔ 04. Task 03 - ICMPv6 echo with large payload PASSED
✔ 04. Task 04 - ICMPv6 echo with iotlab-m3/samr21-xpro 15 minutes PASSED
🟡 04. Task 05 (Experimental) - ICMPv6 multicast echo with samr21-xpro/cc2538 [lille] SKIPPED
🟡 04. Task 06 (Experimental)- ICMPv6 link-local echo with samr21-xpro/cc2538 [lille] SKIPPED
🟡 04. Task 07 (Experimental) - ICMPv6 multicast echo with samr21-xpro/zero + xbee SKIPPED
🟡 04. Task 08 (Experimental) - ICMPv6 echo with samr21-xpro/zero + xbee SKIPPED
✔ 04. Task 09 - ICMPv6 stress test on iotlab-m3 PASSED
✔ 04. Task 10 (Experimental) - ICMPv6 echo with large payload (IPv6 fragmentation) PASSED
✔ 04. Task 11 (Experimental) - ICMPv6 stress test on nrf802154 PASSED
✔ 04. Task 12 (Experimental) - ICMPv6 multicast echo with iotlab-m3/nrf802154 PASSED
✔ 04. Task 13 (Experimental) - ICMPv6 link-local echo with iotlab-m3/nrf802154 PASSED
✔ 05. Task 01 - ICMPv6 echo unicast addresess on native (default route) PASSED
✔ 05. Task 02 - ICMPv6 echo unicast addresess on iotlab-m3 (default route) PASSED
✔ 05. Task 03 - ICMPv6 echo unicast addresess on native (specific route) PASSED
✔ 05. Task 04 - ICMPv6 echo unicast addresess on iotlab-m3 (static route) PASSED
✔ 06. Task 01 - UDP on iotlab-m3 PASSED
✔ 06. Task 02 - UDP on iotlab-m3 (UDP port compression) PASSED
✔ 06. Task 03 - UDP on native (non-existent neighbor) PASSED
✔ 06. Task 04 - UDP on iotlab-m3 (non-existent neighbor) PASSED
✔ 06. Task 05 - Empty UDP on native PASSED
✔ 06. Task 06 - Empty UDP on iotlab-m3 PASSED
✔ 07. Task 01 - ICMPv6 echo on iotlab-m3 with three hops (static route) PASSED
✔ 07. Task 02 - UDP on iotlab-m3 with three hops (static route) PASSED
✔ 07. Task 03 - ICMPv6 echo on iotlab-m3 with three hops (RPL route) PASSED
✔ 07. Task 04 - UDP on iotlab-m3 with three hops (RPL route) PASSED
🟡 07. Task 05 (Experimental) - UDP with large payload on iotlab-m3 with three hops (RPL route) SKIPPED
✔ 08. Task 01 - ICMPv6 echo between native and Linux PASSED
✔ 08. Task 08 - UDP between GNRC and lwIP on iotlab-m3 PASSED
✔ 09. Task 01 - CORD Endpoint PASSED
✔ 09. Task 02 - Confirmable retries [False-0] PASSED
✔ 09. Task 02 - Confirmable retries [True-1] PASSED
✔ 09. Task 03 - Block1 PASSED
✔ 09. Task 04 - Block2 PASSED
✔ 09. Task 05 - Observe registration and notification PASSED
✔ 10. Task 01 - Destination unreachable - no route to destination PASSED
✔ 10. Task 02 - Destination unreachable - Beyond scope of source address PASSED
✔ 10. Task 03 - Destination unreachable - address unreachable (target node address) PASSED
✔ 10. Task 04 - Destination unreachable - address unreachable (neighbor cache miss) PASSED
✔ 10. Task 05 - Destination unreachable - port unreachable PASSED
✔ 10. Task 06 - Destination unreachable - port unreachable (large payload) PASSED
✔ 10. Task 07 - Packet too big PASSED
✔ 10. Task 08 - Time exceeded - hop limit exceeded in transit PASSED
✔ 10. Task 09 - Parameter problem - erroneous header field encountered PASSED
✔ 10. Task 10 - IPv6-in-IPv6 encapsulation PASSED
✔ 11. Task 01 - LoRaWAN example PASSED
✖ 11. Task 02 - OTAA join procedure FAILED
✔ 11. Task 03 - ABP join procedure PASSED
✔ 11. Task 04 - LoRaWAN device parameters persistence PASSED
✔ 11. Task 05 - GNRC LoRaWAN (Over The Air Activation) [otaa] PASSED
🟡 11. Task 06 - GNRC LoRaWAN (ABP) [abp] SKIPPED
miri64 commented 2 years ago

Test Report

Task Outcome
✔ 11. Task 02 - OTAA join procedure [0] PASSED
✔ 11. Task 02 - OTAA join procedure [3] PASSED
✔ 11. Task 02 - OTAA join procedure [5] PASSED
✔ 11. Task 03 - ABP join procedure [0] PASSED
✔ 11. Task 03 - ABP join procedure [3] PASSED
✔ 11. Task 03 - ABP join procedure [5] PASSED
🟡 11. Task 06 - GNRC LoRaWAN (ABP) [abp] SKIPPED
miri64 commented 2 years ago

Ran those with #255 which parametrized the DR and hardens the join tests in general.

chrysn commented 2 years ago

Closing in favor of #257