search

RIOT-OS / Release-Specs

Specification for RIOT releases and corresponding test configurations
4 stars 21 forks source link

Release 2021.07 - RC3 #232

Closed MrKevinWeiss closed 3 years ago

MrKevinWeiss commented 3 years ago

This issue lists the status of all tests for the Release Candidate 3 of the 2021.07 release.

Specs tested:

riot-ci commented 3 years ago

Test Report

Task Outcome
✔ 01. Task 02 - Unittests on native PASSED
✔ 01. Task 03 - Unittests on native separated [tests-analog_util] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-base64] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-bcd] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-bitfield] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-bloom] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-bluetil] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-checksum] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-clif] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-color] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-core] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-credman] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-div] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-ecc] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-fib] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-fib_sr] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-fmt] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-frac] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-gcoap] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-gnrc_ipv6] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-gnrc_ipv6_hdr] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-gnrc_ipv6_nib] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-gnrc_mac_internal] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-gnrc_netif_pktq] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-gnrc_sixlowpan_frag_vrb] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-gnrc_udp] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-hashes] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-ieee802154] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-inet_csum] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-ipv4_addr] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-ipv6_addr] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-ipv6_hdr] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-luid] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-matstat] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-mtd] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-nanocoap] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-netopt] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-netreg] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-phydat] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-pkt] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-pktbuf] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-pktqueue] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-printf_float] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-priority_pktqueue] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-rtc] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-rtt_rtc] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-saul_reg] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-scanf_float] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-seq] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-sht1x] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-sixlowpan] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-sixlowpan_ctx] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-sixlowpan_sfr] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-sock_util] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-timex] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-tsrb] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-turo] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-uri_parser] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-uuid] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-vfs] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-zptr] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-ztimer] PASSED
✔ 03. Task 01 - ICMPv6 multicast echo on native PASSED
✔ 03. Task 02 - ICMPv6 link-local echo on native PASSED
✔ 03. Task 03 - ICMPv6 link-local echo on native (1 hour) PASSED
✔ 03. Task 04 - ICMPv6 stress test on native (1 hour) PASSED
🟡 03. Task 05 - ICMPv6 stress test on native (neighbor cache stress) SKIPPED
✔ 03. Task 06 - ICMPv6 link-local echo on native (IPv6 fragmentation) PASSED
✔ 04. Task 01 - ICMPv6 link-local echo with iotlab-m3 PASSED
✔ 04. Task 03 - ICMPv6 echo with large payload PASSED
✔ 04. Task 04 - ICMPv6 echo with iotlab-m3/samr21-xpro 15 minutes PASSED
✖ 04. Task 05 (Experimental) - ICMPv6 multicast echo with samr21-xpro/cc2538 [lille] FAILED
✖ 04. Task 06 (Experimental)- ICMPv6 link-local echo with samr21-xpro/cc2538 [lille] FAILED
🟡 04. Task 07 (Experimental) - ICMPv6 multicast echo with samr21-xpro/zero + xbee SKIPPED
🟡 04. Task 08 (Experimental) - ICMPv6 echo with samr21-xpro/zero + xbee SKIPPED
✔ 04. Task 09 - ICMPv6 stress test on iotlab-m3 PASSED
✔ 04. Task 10 (Experimental) - ICMPv6 echo with large payload (IPv6 fragmentation) PASSED
✔ 04. Task 11 (Experimental) - ICMPv6 stress test on nrf802154 PASSED
✔ 04. Task 12 (Experimental) - ICMPv6 multicast echo with iotlab-m3/nrf802154 PASSED
✔ 04. Task 13 (Experimental) - ICMPv6 link-local echo with iotlab-m3/nrf802154 PASSED
✔ 05. Task 01 - ICMPv6 echo unicast addresess on native (default route) PASSED
✔ 05. Task 02 - ICMPv6 echo unicast addresess on iotlab-m3 (default route) PASSED
✔ 05. Task 03 - ICMPv6 echo unicast addresess on native (specific route) PASSED
✔ 05. Task 04 - ICMPv6 echo unicast addresess on iotlab-m3 (static route) PASSED
✔ 06. Task 01 - UDP on iotlab-m3 PASSED
✔ 06. Task 03 - UDP on native (non-existent neighbor) PASSED
✔ 06. Task 04 - UDP on iotlab-m3 (non-existent neighbor) PASSED
✔ 06. Task 05 - Empty UDP on native PASSED
✔ 07. Task 01 - ICMPv6 echo on iotlab-m3 with three hops (static route) PASSED
✔ 07. Task 02 - UDP on iotlab-m3 with three hops (static route) PASSED
✔ 07. Task 03 - ICMPv6 echo on iotlab-m3 with three hops (RPL route) PASSED
✔ 07. Task 04 - UDP on iotlab-m3 with three hops (RPL route) PASSED
🟡 07. Task 05 (Experimental) - UDP with large payload on iotlab-m3 with three hops (RPL route) SKIPPED
✔ 08. Task 01 - ICMPv6 echo between native and Linux PASSED
✔ 08. Task 08 - UDP between GNRC and lwIP on iotlab-m3 PASSED
✔ 09. Task 01 - CORD Endpoint PASSED
✔ 09. Task 02 - Confirmable retries [False-0] PASSED
✔ 09. Task 02 - Confirmable retries [True-1] PASSED
✔ 09. Task 03 - Block1 PASSED
✔ 09. Task 04 - Block2 PASSED
✔ 09. Task 05 - Observe registration and notification PASSED
✔ 10. Task 01 - Destination unreachable - no route to destination PASSED
✔ 10. Task 02 - Destination unreachable - Beyond scope of source address PASSED
✔ 10. Task 03 - Destination unreachable - address unreachable (target node address) PASSED
✔ 10. Task 04 - Destination unreachable - address unreachable (neighbor cache miss) PASSED
✔ 10. Task 05 - Destination unreachable - port unreachable PASSED
✔ 10. Task 06 - Destination unreachable - port unreachable (large payload) PASSED
✔ 10. Task 07 - Packet too big PASSED
✔ 10. Task 09 - Parameter problem - erroneous header field encountered PASSED
✔ 10. Task 10 - IPv6-in-IPv6 encapsulation PASSED
✖ 11. Task 05 - GNRC LoRaWAN (Over The Air Activation) [otaa] FAILED
✖ 11. Task 06 - GNRC LoRaWAN (ABP) [abp] FAILED
MrKevinWeiss commented 3 years ago

02 was run on the last PR that was merged.

MrKevinWeiss commented 3 years ago

The only tests that I think should be rerun would be LoRaWAN but given the challenge in setting them up I could also let it go... Would you be able to rerun them @jia200x ?

jia200x commented 3 years ago

All LoRaWAN test pass:

Task #01 - LoRaWAN example

PASSED ``` 2021-07-15 15:21:14,326 # Join procedure succeeded 2021-07-15 15:21:14,328 # Sending: This is RIOT! 2021-07-15 15:21:35,439 # Sending: This is RIOT! 2021-07-15 15:21:56,552 # Sending: This is RIOT! 2021-07-15 15:22:17,665 # Sending: This is RIOT! 2021-07-15 15:22:38,777 # Sending: This is RIOT! ```

Task #02 - OTAA join procedure

PASSED ``` loramac set dr 0 2021-07-15 16:50:51,588 # loramac set dr 0 loramac tx RIOT cnf 123 2021-07-15 16:51:00,012 # loramac tx RIOT cnf 123 2021-07-15 16:51:00,014 # Cannot send: not joined > loramajoin otaa 2021-07-15 16:51:04,611 # loramac join otaa 2021-07-15 16:51:12,982 # Join procedure succeeded! > loramatx RIOT cnf 123 2021-07-15 16:51:18,415 # loramac tx RIOT cnf 123 2021-07-15 16:51:21,952 # Received ACK from network 2021-07-15 16:51:21,954 # Message sent with success > loramac tx RIOT uncnf 42 2021-07-15 16:52:02,242 # loramac tx RIOT uncnf 42 2021-07-15 16:52:02,246 # Cannot send: dutycycle restriction > loramac tx RIOT uncnf 42 2021-07-15 16:52:28,583 # loramac tx RIOT uncnf 42 2021-07-15 16:52:28,587 # Cannot send: dutycycle restriction > loramac tx RIOT uncnf 42 2021-07-15 16:53:02,372 # loramac tx RIOT uncnf 42 2021-07-15 16:53:02,376 # Cannot send: dutycycle restriction > loramac tx RIOT uncnf 42 2021-07-15 16:53:40,346 # loramac tx RIOT uncnf 42 2021-07-15 16:53:44,378 # Message sent with success > 2021-07-15 16:54:39,883 # loramac set dr 3 loramac join otaa 2021-07-15 16:54:45,104 # loramac join otaa 2021-07-15 16:54:50,610 # Join procedure succeeded! loramac tx RIOT cnf 123 2021-07-15 16:54:59,914 # loramac tx RIOT cnf 123 2021-07-15 16:55:01,261 # Received ACK from network 2021-07-15 16:55:01,263 # Message sent with success loramac tx RIOT uncnf 42 2021-07-15 16:55:16,859 # loramac tx RIOT uncnf 42 2021-07-15 16:55:19,891 # Message sent with success > 2021-07-15 16:55:47,918 # loramac set dr 5 loramac join otaa 2021-07-15 16:55:50,143 # loramac join otaa 2021-07-15 16:55:55,328 # Join procedure succeeded! loramac tx RIOT cnf 123 2021-07-15 16:57:00,116 # loramac tx RIOT cnf 123 2021-07-15 16:57:01,245 # Received ACK from network 2021-07-15 16:57:01,247 # Message sent with success loramac tx RIOT uncnf 42 2021-07-15 16:57:08,978 # loramac tx RIOT uncnf 42 2021-07-15 16:57:12,011 # Message sent with success > ```

Task #03 - ABP join procedure

PASSED ``` > loramac join abp 2021-07-15 17:00:24,949 # loramac join abp 2021-07-15 17:00:24,952 # Join procedure succeeded! > loramac tx RIOT cnf 123 2021-07-15 17:00:33,577 # loramac tx RIOT cnf 123 2021-07-15 17:02:45,948 # Received ACK from network 2021-07-15 17:02:45,950 # Message sent with success > loramac tx RIOT uncnf 42 2021-07-15 17:05:57,126 # loramac tx RIOT uncnf 42 2021-07-15 17:06:01,158 # Message sent with success loramac get dr 2021-07-15 17:06:55,663 # loramac get dr 2021-07-15 17:06:55,664 # DATARATE: 0 2021-07-15 17:20:52,161 # loramac set dr 3 loramac join abp 2021-07-15 17:21:03,884 # loramac join abp 2021-07-15 17:21:03,887 # Join procedure succeeded! loramac tx RIOT cnf 123 2021-07-15 17:21:09,103 # loramac tx RIOT cnf 123 2021-07-15 17:21:10,451 # Received ACK from network 2021-07-15 17:21:10,453 # Message sent with success > loramac tx RIOT uncnf 42 2021-07-15 17:21:19,453 # loramac tx RIOT uncnf 42 2021-07-15 17:21:19,457 # Cannot send: dutycycle restriction > loramac tx RIOT uncnf 42 2021-07-15 17:21:43,584 # loramac tx RIOT uncnf 42 2021-07-15 17:21:46,616 # Message sent with success > 2021-07-15 17:22:25,663 # loramac set dr 5 loramac join abp 2021-07-15 17:22:34,978 # loramac join abp 2021-07-15 17:22:34,980 # Join procedure succeeded! loramac tx RIOT cnf 123 2021-07-15 17:22:39,036 # loramac tx RIOT cnf 123 2021-07-15 17:22:40,164 # Received ACK from network 2021-07-15 17:22:40,166 # Message sent with success loramac tx RIOT uncnf 42 2021-07-15 17:24:46,021 # loramac tx RIOT uncnf 42 2021-07-15 17:24:49,053 # Message sent with success > ```

Task #04 - LoRaWAN device parameters persistence

PASSED ``` 2021-07-15 15:25:23,707 # reboot 2021-07-15 15:25:23,794 # main(): This is RIOT! (Version: 2021.10-devel-23-g1a4b3-HEAD) 2021-07-15 15:25:23,797 # All up, running the shell now > oramac get deveui 2021-07-15 15:25:31,035 # oramac get deveui 2021-07-15 15:25:31,038 # shell: command not found: oramac > loramac get deveui 2021-07-15 15:25:34,581 # loramac get deveui 2021-07-15 15:25:34,583 # DEVEUI: 0000000000000000 > loramac get appeui 2021-07-15 15:25:36,772 # loramac get appeui 2021-07-15 15:25:36,774 # APPEUI: 0000000000000000 > loramac get appkey 2021-07-15 15:25:38,876 # loramac get appkey 2021-07-15 15:25:38,880 # APPKEY: 00000000000000000000000000000000 > loramac get devaddr 2021-07-15 15:25:44,741 # loramac get devaddr 2021-07-15 15:25:44,742 # DEVADDR: 00000000 > loramac get appskey 2021-07-15 15:25:47,869 # loramac get appskey 2021-07-15 15:25:47,873 # APPSKEY: 00000000000000000000000000000000 > loramac get nwkskey 2021-07-15 15:25:52,038 # loramac get nwkskey 2021-07-15 15:25:52,042 # NWKSKEY: 00000000000000000000000000000000 loramac set deveui 001122334455667777 2021-07-15 15:26:02,903 # loramac set deveui 0011223344556677 loramac set appeui 001122334455667777 2021-07-15 15:26:08,183 # loramac set appeui 0011223344556677 loramac set appkey 0011223344556677001122334455667777 2021-07-15 15:26:14,986 # loramac set appkey 00112233445566770011223344556677 loramac saveve 2021-07-15 15:26:21,252 # loramac save rebootot 2021-07-15 15:26:26,382 # reboot 2021-07-15 15:26:26,470 # main(): This is RIOT! (Version: 2021.10-devel-23-g1a4b3-HEAD) 2021-07-15 15:26:26,473 # All up, running the shell now loramac get deveuiui 2021-07-15 15:27:05,574 # loramac get deveui 2021-07-15 15:27:05,576 # DEVEUI: 0011223344556677 > loramac get appeui 2021-07-15 15:27:09,480 # loramac get appeui 2021-07-15 15:27:09,483 # APPEUI: 0011223344556677 > loramac get appkey 2021-07-15 15:27:12,334 # loramac get appkey 2021-07-15 15:27:12,338 # APPKEY: 00112233445566770011223344556677 loramac erasese 2021-07-15 15:27:19,865 # loramac erase loramac set devaddr 0011223333 2021-07-15 15:27:29,441 # loramac set devaddr 00112233 loramac set appskey 0011223344556677001122334455667777 2021-07-15 15:27:35,099 # loramac set appskey 00112233445566770011223344556677 loramac set nwkskey 0011223344556677001122334455667777 2021-07-15 15:27:40,861 # loramac set nwkskey 00112233445566770011223344556677 loramac saveve 2021-07-15 15:27:46,008 # loramac save > reboot 2021-07-15 15:27:47,847 # reboot 2021-07-15 15:27:47,935 # main(): This is RIOT! (Version: 2021.10-devel-23-g1a4b3-HEAD) 2021-07-15 15:27:47,938 # All up, running the shell now loramac get devaddr 2021-07-15 15:35:24,351 # loramac get devaddr 2021-07-15 15:35:24,353 # DEVADDR: 00112233 > loramac get nwkskey 2021-07-15 15:35:30,535 # loramac get nwkskey 2021-07-15 15:35:30,539 # NWKSKEY: 00112233445566770011223344556677 > loramac get appskey 2021-07-15 15:35:32,839 # loramac get appskey 2021-07-15 15:35:32,843 # APPSKEY: 00112233445566770011223344556677 > ```

Task #05 - GNRC LoRaWAN (Over The Air Activation)

PASSED ``` 2021-07-15 17:36:55,679 # ifconfig 3 set dr 5 2021-07-15 17:36:55,683 # success: set datarate on interface 3 to 5 ifconfig 3 up 2021-07-15 17:37:02,082 # ifconfig 3 up > ifconfig 2021-07-15 17:37:09,315 # ifconfig 2021-07-15 17:37:09,326 # Iface 3 HWaddr: 00:B9:8D:22 Frequency: 868099975Hz RSSI: 99 BW: 125kHz SF: 7 CR: 4/5 Link: up 2021-07-15 17:37:09,333 # TX-Power: 14dBm State: SLEEP Demod margin.: 0 Num gateways.: 0 2021-07-15 17:37:09,335 # IQ_INVERT 2021-07-15 17:37:09,339 # RX_SINGLE OTAA L2-PDU:767 2021-07-15 17:37:09,340 # > txtsnd 3 01 RIOT 2021-07-15 17:38:03,458 # txtsnd 3 01 RIOT > ifconfig 3 ack_req 2021-07-15 17:38:19,549 # ifconfig 3 ack_req 2021-07-15 17:38:19,551 # success: set option txtsnd 3 01 RIOTeq 2021-07-15 17:38:21,812 # txtsnd 3 01 RIOT > txtsnd 3 01 RIOT 2021-07-15 17:39:27,588 # txtsnd 3 01 RIOT > 2021-07-15 17:39:28,699 # PKTDUMP: data received: 2021-07-15 17:39:28,704 # ~~ SNIP 0 - size: 5 byte, type: NETTYPE_UNDEF (0) 2021-07-15 17:39:28,707 # 00000000 48 4F 4C 41 0A 2021-07-15 17:39:28,712 # ~~ SNIP 1 - size: 9 byte, type: NETTYPE_NETIF (-1) 2021-07-15 17:39:28,715 # if_pid: 3 rssi: -32768 lqi: 0 2021-07-15 17:39:28,716 # flags: 0x0 2021-07-15 17:39:28,717 # src_l2addr: (nil) 2021-07-15 17:39:28,718 # dst_l2addr: 02 2021-07-15 17:39:28,722 # ~~ PKT - 2 snips, total size: 14 byte txtsnd 3 01 RIOT 2021-07-15 17:39:34,403 # txtsnd 3 01 RIOT ```

Task #06 - GNRC LoRaWAN (ABP)

PASSED ``` 2021-07-15 17:48:11,584 # ifconfig 3 -otaa 2021-07-15 17:48:11,586 # success: unset option iconfig 3 up 2021-07-15 17:48:17,328 # iconfig 3 up 2021-07-15 17:48:17,331 # shell: command not found: iconfig ifconfig 3 up 2021-07-15 17:48:24,569 # ifconfig 3 up > ifconfig 2021-07-15 17:48:26,639 # ifconfig 2021-07-15 17:48:26,648 # Iface 3 HWaddr: 00:8F:96:3A Frequency: 868299987Hz RSSI: 99 BW: 125kHz SF: 7 CR: 4/5 Link: up 2021-07-15 17:48:26,656 # TX-Power: 14dBm State: SLEEP Demod margin.: 0 Num gateways.: 0 2021-07-15 17:48:26,659 # L2-PDU:767 2021-07-15 17:48:26,659 # > txtsnd 3 01 RIOT 2021-07-15 17:48:32,798 # txtsnd 3 01 RIOT > txtsnd 3 01 RIOT 2021-07-15 17:48:38,783 # txtsnd 3 01 RIOT > ifconfig 3 set dr 5 2021-07-15 17:48:50,710 # ifconfig 3 set dr 5 2021-07-15 17:48:50,714 # success: set datarate on interface 3 to 5 ifconfig 3 -ack_req 2021-07-15 17:49:25,889 # ifconfig 3 -ack_req 2021-07-15 17:49:25,891 # success: unset option txtsnd 3 01 RIOT 5 2021-07-15 17:49:28,545 # txtsnd 3 01 RIOT > 2021-07-15 17:49:29,656 # PKTDUMP: data received: 2021-07-15 17:49:29,660 # ~~ SNIP 0 - size: 5 byte, type: NETTYPE_UNDEF (0) 2021-07-15 17:49:29,663 # 00000000 48 4F 4C 41 0A 2021-07-15 17:49:29,668 # ~~ SNIP 1 - size: 9 byte, type: NETTYPE_NETIF (-1) 2021-07-15 17:49:29,671 # if_pid: 3 rssi: -32768 lqi: 0 2021-07-15 17:49:29,672 # flags: 0x0 2021-07-15 17:49:29,673 # src_l2addr: (nil) 2021-07-15 17:49:29,675 # dst_l2addr: 02 2021-07-15 17:49:29,679 # ~~ PKT - 2 snips, total size: 14 byte ifconfig 3 ack_req 2021-07-15 17:49:35,011 # ifconfig 3 ack_req 2021-07-15 17:49:35,012 # success: set option txtsnd 3 01 RIOT 2021-07-15 17:49:38,125 # txtsnd 3 01 RIOT ```
MrKevinWeiss commented 3 years ago

Tests rerun, I think everything is good, we are ready for a release!