Release 2021.04 - RC3 - Githubissues

kaspar030 commented 3 years ago

This issue lists the status of all tests for the Release Candidate 3 of the 2021.04 release.

Specs tested:

[x] 01-ci
- [x] Task #01 - Compile test Done by murdock
- [x] Task #02 - Unittests on native @riot-ci (see https://github.com/RIOT-OS/Release-Specs/issues/219#issuecomment-828418046)
- [x] Task #03 - Unittests on native separated @riot-ci (see https://github.com/RIOT-OS/Release-Specs/issues/219#issuecomment-828418136)
- [x] Task #04 - Unittests on iotlab-m3 @riot-ci (see https://github.com/RIOT-OS/Release-Specs/issues/219#issuecomment-828419348)
[x] 02-tests
- [x] Task #01 - All tests on native done by Murdock
- [x] Task #02 - Subset of tests on iotlab-m3
- [x] Task #03 - Subset of tests on samr21-xpro
[ ] 03-single-hop-ipv6-icmp
- [x] Task #01 - ICMPv6 multicast echo on native @riot-ci (see https://github.com/RIOT-OS/Release-Specs/issues/219#issuecomment-828423579)
- [x] Task #02 - ICMPv6 link-local echo on native @riot-ci (see https://github.com/RIOT-OS/Release-Specs/issues/219#issuecomment-828424990)
- [x] Task #03 - ICMPv6 link-local echo on native (1 hour) @riot-ci
- [x] Task #04 - ICMPv6 stress test on native (1 hour) @riot-ci (see https://github.com/RIOT-OS/Release-Specs/issues/219#issuecomment-828515325)
- [ ] Task #05 - ICMPv6 stress test on native (neighbor cache stress) skipped, known bug: https://github.com/RIOT-OS/RIOT/issues/12565
- [x] Task #06 - ICMPv6 link-local echo on native (IPv6 fragmentation) @riot-ci (see https://github.com/RIOT-OS/Release-Specs/issues/219#issuecomment-828518062)
[ ] 04-single-hop-6lowpan-icmp
[x] 05-single-hop-route
[x] 06-single-hop-udp
- [x] Task #01 - UDP on iotlab-m3 @riot-ci (see https://github.com/RIOT-OS/Release-Specs/issues/219#issuecomment-828476787)
- [x] Task #02 - UDP on iotlab-m3 (UDP port compression) @riot-ci (see https://github.com/RIOT-OS/Release-Specs/issues/219#issuecomment-828506031)
- [x] Task #03 - UDP on native (non-existent neighbor) @riot-ci (see https://github.com/RIOT-OS/Release-Specs/issues/219#issuecomment-828519368)
- [x] Task #04 - UDP on iotlab-m3 (non-existent neighbor) @riot-ci (see https://github.com/RIOT-OS/Release-Specs/issues/219#issuecomment-828506885)
- [x] Task #05 - Empty UDP on native @riot-ci (see https://github.com/RIOT-OS/Release-Specs/issues/219#issuecomment-828519968)
- [x] Task #06 - Empty UDP on iotlab-m3 @riot-ci (see https://github.com/RIOT-OS/Release-Specs/issues/219#issuecomment-828508053)
[ ] 07-multi-hop
[x] 08-interop
- [x] Task #01 - ICMPv6 echo between native and Linux @riot-ci (see https://github.com/RIOT-OS/Release-Specs/issues/219#issuecomment-828520322)
- [x] Task #02 - ICMPv6 echo between iotlab-m3 and Linux with 6LowPAN @kaspar030 (see https://github.com/RIOT-OS/Release-Specs/issues/219#issuecomment-831206173)
- [x] Task #03 - UDP exchange between iotlab-m3 and Contiki
- [x] Task #04 - ICMPv6 echo between iotlab-m3 and Internet host through Linux with 6LowPAN @kaspar030
- [x] Task #05 - ICMPv6 echo between iotlab-m3 and Internet host through RIOT border router
- [x] Task #06 - UDP between iotlab-m3 and Internet host through RIOT border router
- [x] Task #07 - UDP between iotlab-m3 and Internet host through RIOT border router (200b payload)
- [x] Task #08 - UDP between GNRC and lwIP on iotlab-m3 @riot-ci (see https://github.com/RIOT-OS/Release-Specs/issues/219#issuecomment-828520440)
- [x] ~~Task #09 - deprecated~~
- [x] ~~Task #10 - deprecated~~
- [x] Task #11 - UDP exchange between iotlab-m3 and Zephyr @leandrolanzieri (see https://github.com/RIOT-OS/Release-Specs/issues/219#issuecomment-829992107)
- [x] Task #12 - GNRC Border Router with WiFi uplink to the Internet
- [x] Task #13 - GNRC Networking
[x] 09-coap
- [x] Task #01 - CORD Endpoint @riot-ci (see https://github.com/RIOT-OS/Release-Specs/issues/219#issuecomment-828521419)
- [x] Task #02 - Confirmable retries @riot-ci (see https://github.com/RIOT-OS/Release-Specs/issues/219#issuecomment-828521734)
- [x] Task #03 - Block1 @riot-ci (see https://github.com/RIOT-OS/Release-Specs/issues/219#issuecomment-828524848)
- [x] Task #04 - Block2 @riot-ci (see https://github.com/RIOT-OS/Release-Specs/issues/219#issuecomment-828526761)
- [x] Task #05 - Observe registration and notification @riot-ci (see https://github.com/RIOT-OS/Release-Specs/issues/219#issuecomment-828526905)
[x] 10-icmpv6-error
[x] 11-lorawan
- [x] Task #01 - LoRaWAN example @fjmolinas (see https://github.com/RIOT-OS/Release-Specs/issues/219#issuecomment-828471728)
- [x] Task #02 - OTAA join procedure @fjmolinas (see https://github.com/RIOT-OS/Release-Specs/issues/219#issuecomment-828475402)
- [x] Task #03 - ABP join procedure @fjmolinas (see https://github.com/RIOT-OS/Release-Specs/issues/219#issuecomment-828478891)
- [x] Task #04 - LoRaWAN device parameters persistence @fjmolinas (see https://github.com/RIOT-OS/Release-Specs/issues/219#issuecomment-828469306)
- [x] Task #05 - GNRC LoRaWAN (Over The Air Activation) @jia200x (see https://github.com/RIOT-OS/Release-Specs/issues/219#issuecomment-829354169)
- [x] Task #06 - GNRC LoRaWAN (ABP) @fjmolinas (see https://github.com/RIOT-OS/Release-Specs/issues/219#issuecomment-828481635)
[ ] 99-compile-and-test-one-board
- [ ] Task #01 - Run tests on different hardwares

FAILED

###### Failures ``` riot_ctrl = .ctrl at 0x7f3349e509d0>, ttn_client = dev_id = 'riot-examples-lorawan-otaa', deveui = '00D23F63A8D87783', appeui = '70B3D57ED00204EC' appkey = '0A3E1510AEB8DB583B10DCA691522409' @pytest.mark.iotlab_creds # nodes passed to riot_ctrl fixture @pytest.mark.parametrize('nodes,dev_id', [pytest.param(['b-l072z-lrwan1'], "otaa")], indirect=['nodes', 'dev_id']) # pylint: disable=R0913 def test_task05(riot_ctrl, ttn_client, dev_id, deveui, appeui, appkey): node = riot_ctrl(0, GNRC_LORAWAN_APP, ShellGnrcLoRaWAN) iface = lorawan_netif(node) assert iface # Set the OTAA keys node.ifconfig_set(iface, "deveui", deveui) node.ifconfig_set(iface, "appeui", appeui) node.ifconfig_set(iface, "appkey", appkey) # Enable OTAA node.ifconfig_flag(iface, "otaa", enable=True) # Trigger Join Request node.ifconfig_up(iface) # Wait until the LoRaWAN network is joined time.sleep(10) netif = ifconfig(node, iface) assert netif[str(iface)]["link"] == "up" > run_lw_test(node, ttn_client, iface, dev_id) 11-lorawan/test_spec11.py:199: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ node = , ttn_client = iface = 3, dev_id = 'riot-examples-lorawan-otaa' def run_lw_test(node, ttn_client, iface, dev_id): # Disable confirmable messages node.ifconfig_flag(iface, "ack_req", enable=False) # Push a downlink message to the TTN server dl_data = {"payload_raw": DOWNLINK_PAYLOAD, "port": APP_PORT, "confirmed": True} ttn_client.publish_to_dev(dev_id, **dl_data) # Send a message. The send function will return True if the downlink is # receives (as expected) > assert node.send(iface, APP_PAYLOAD) is True E AssertionError: assert False is True E + where False = .wrapper of >(3, 'This is RIOT!') E + where .wrapper of > = .send 11-lorawan/test_spec11.py:47: AssertionError ``` ###### Captured stdout call ``` Building application "gnrc_lorawan" for "b-l072z-lrwan1" with MCU "stm32". "make" -C /home/francisco/workspace/RIOT/boards/b-l072z-lrwan1 "make" -C /home/francisco/workspace/RIOT/core "make" -C /home/francisco/workspace/RIOT/cpu/stm32 "make" -C /home/francisco/workspace/RIOT/cpu/cortexm_common "make" -C /home/francisco/workspace/RIOT/cpu/cortexm_common/periph "make" -C /home/francisco/workspace/RIOT/cpu/stm32/periph "make" -C /home/francisco/workspace/RIOT/cpu/stm32/stmclk "make" -C /home/francisco/workspace/RIOT/cpu/stm32/vectors "make" -C /home/francisco/workspace/RIOT/drivers "make" -C /home/francisco/workspace/RIOT/drivers/netdev "make" -C /home/francisco/workspace/RIOT/drivers/periph_common "make" -C /home/francisco/workspace/RIOT/drivers/sx127x "make" -C /home/francisco/workspace/RIOT/sys "make" -C /home/francisco/workspace/RIOT/sys/auto_init "make" -C /home/francisco/workspace/RIOT/sys/crypto "make" -C /home/francisco/workspace/RIOT/sys/div "make" -C /home/francisco/workspace/RIOT/sys/fmt "make" -C /home/francisco/workspace/RIOT/sys/frac "make" -C /home/francisco/workspace/RIOT/sys/hashes "make" -C /home/francisco/workspace/RIOT/sys/iolist "make" -C /home/francisco/workspace/RIOT/sys/isrpipe "make" -C /home/francisco/workspace/RIOT/sys/luid "make" -C /home/francisco/workspace/RIOT/sys/malloc_thread_safe "make" -C /home/francisco/workspace/RIOT/sys/net/gnrc "make" -C /home/francisco/workspace/RIOT/sys/net/gnrc/link_layer/lorawan "make" -C /home/francisco/workspace/RIOT/sys/net/gnrc/netapi "make" -C /home/francisco/workspace/RIOT/sys/net/gnrc/netif "make" -C /home/francisco/workspace/RIOT/sys/net/gnrc/netif/hdr "make" -C /home/francisco/workspace/RIOT/sys/net/gnrc/netif/init_devs "make" -C /home/francisco/workspace/RIOT/sys/net/gnrc/netif/lorawan "make" -C /home/francisco/workspace/RIOT/sys/net/gnrc/netreg "make" -C /home/francisco/workspace/RIOT/sys/net/gnrc/pkt "make" -C /home/francisco/workspace/RIOT/sys/net/gnrc/pktbuf "make" -C /home/francisco/workspace/RIOT/sys/net/gnrc/pktbuf_static "make" -C /home/francisco/workspace/RIOT/sys/net/gnrc/pktdump "make" -C /home/francisco/workspace/RIOT/sys/net/link_layer/eui_provider "make" -C /home/francisco/workspace/RIOT/sys/net/link_layer/l2util "make" -C /home/francisco/workspace/RIOT/sys/net/netif "make" -C /home/francisco/workspace/RIOT/sys/newlib_syscalls_default "make" -C /home/francisco/workspace/RIOT/sys/od "make" -C /home/francisco/workspace/RIOT/sys/pm_layered "make" -C /home/francisco/workspace/RIOT/sys/random "make" -C /home/francisco/workspace/RIOT/sys/random/tinymt32 "make" -C /home/francisco/workspace/RIOT/sys/shell "make" -C /home/francisco/workspace/RIOT/sys/shell/commands "make" -C /home/francisco/workspace/RIOT/sys/stdio_uart "make" -C /home/francisco/workspace/RIOT/sys/tsrb "make" -C /home/francisco/workspace/RIOT/sys/xtimer "make" -C /home/francisco/workspace/RIOT/sys/ztimer text data bss dec hex filename 55292 196 6484 61972 f214 /home/francisco/workspace/RIOT/examples/gnrc_lorawan/bin/b-l072z-lrwan1/gnrc_lorawan.elf iotlab-node --jmespath='keys(@)[0]' --format='lambda ret: exit(int(ret))' --id 261526 --list saclay,st-lrwan1,12 --flash /home/francisco/workspace/RIOT/examples/gnrc_lorawan/bin/b-l072z-lrwan1/gnrc_lorawan.bin ssh -t molina@saclay.iot-lab.info 'socat - tcp:st-lrwan1-12.saclay.iot-lab.info:20000' main(): This is RIOT! (Version: 2021.07-devel-21-g26db4f-HEAD) Initialization successful - starting the shell now > NETOPT_RX_END_IRQ not implemented by driver > ifconfig ifconfig Iface 3 HWaddr: 00:00:00:00 Frequency: 868299987Hz RSSI: 99 BW: 125kHz SF: 7 CR: 4/5 Link: down TX-Power: 14dBm State: SLEEP Demod margin.: 0 Num gateways.: 0 OTAA L2-PDU:7167 > ifconfig 3 set deveui 00D23F63A8D87783 ifconfig 3 set deveui 00D23F63A8D87783 success: set long address on interface 3 to 00D23F63A8D87783 > ifconfig 3 set appeui 70B3D57ED00204EC ifconfig 3 set appeui 70B3D57ED00204EC success: set AppEUI on interface 3 to 70B3D57ED00204EC > ifconfig 3 set appkey 0A3E1510AEB8DB583B10DCA691522409 ifconfig 3 set appkey 0A3E1510AEB8DB583B10DCA691522409 success: set AppKey on interface 3 to 0A3E1510AEB8DB583B10DCA691522409 > ifconfig 3 otaa ifconfig 3 otaa success: set option > ifconfig 3 up ifconfig 3 up > ifconfig 3 ifconfig 3 Iface 3 HWaddr: 26:01:4E:74 Frequency: 869524963Hz RSSI: 99 BW: 125kHz SF: 12 CR: 4/5 Link: up TX-Power: 14dBm State: SLEEP Demod margin.: 0 Num gateways.: 0 IQ_INVERT RX_SINGLE OTAA L2-PDU:2047 > ifconfig 3 -ack_req ifconfig 3 -ack_req success: unset option > send 3 "This is RIOT!" send 3 "This is RIOT!" Successfully sent packet > ```