search

RIOT-OS / Release-Specs

Specification for RIOT releases and corresponding test configurations
4 stars 21 forks source link

Automate spec08 and task11, zephyr interop test #295

Open MrKevinWeiss opened 11 months ago

MrKevinWeiss commented 11 months ago

Here it is, the next step in automation. Ideally https://github.com/zephyrproject-rtos/zephyr/pull/65781 gets merged in and we don't need the configuration step. I also don't know if it would be better to fix our test to a version or test with master (or both).

Either way, local testing worked:

``` test: commands[0]> pytest -k 'spec08 and task11' --capture=tee-sys ============================= test session starts ============================== platform linux -- Python 3.10.12, pytest-7.3.2, pluggy-1.3.0 -- /home/weiss/repos/Release-Specs/.tox/test/bin/python cachedir: .tox/test/.pytest_cache rootdir: /home/weiss/repos/Release-Specs configfile: setup.cfg plugins: rerunfailures-12.0, cov-4.1.0 collecting ... collected 137 items / 136 deselected / 1 selected 08-interop/test_spec08.py::test_task11[nodes0] Openbox-Message: Unable to find a valid menu file "/var/lib/openbox/debian-menu.xml" Xlib: extension "DPMS" missing on display ":0". The VNC desktop is: d59bc585dfec:0 PORT=5900 FATAL ERROR: already initialized in /workdir, aborting. Note: In your environment, ZEPHYR_BASE is set to: /workdir/zephyr This forces west to search for a workspace there. Try unsetting ZEPHYR_BASE and re-running this command. === updating cmsis (modules/hal/cmsis): HEAD is now at 4b96cbb doc: Update documentations for CMSIS 5.9.0 === updating hal_atmel (modules/hal/atmel): HEAD is now at 942d664 sam: Add missing REG_RTC_WPMR register instances Zephyr (/workdir/zephyr/share/zephyr-package/cmake) has been added to the user package registry in: ~/.cmake/packages/Zephyr ZephyrUnittest (/workdir/zephyr/share/zephyrunittest-package/cmake) has been added to the user package registry in: ~/.cmake/packages/ZephyrUnittest Defaulting to user installation because normal site-packages is not writeable ERROR: Could not open requirements file: [Errno 2] No such file or directory: '/home/user/zephyrproject/zephyr/scripts/requirements.txt' -- west build: generating a build system Loading Zephyr default modules (Zephyr base (cached)). -- Application: /workdir/zephyr/samples/net/sockets/echo_server -- CMake version: 3.22.1 -- Cache files will be written to: /home/user/.cache/zephyr -- Zephyr version: 3.5.99 (/workdir/zephyr) -- Found west (found suitable version "1.2.0", minimum required is "0.14.0") -- Board: atsamr21_xpro -- Found host-tools: zephyr 0.16.4 (/opt/toolchains/zephyr-sdk-0.16.4) -- Found toolchain: zephyr 0.16.4 (/opt/toolchains/zephyr-sdk-0.16.4) -- Found BOARD.dts: /workdir/zephyr/boards/arm/atsamr21_xpro/atsamr21_xpro.dts -- Generated zephyr.dts: /workdir/zephyr/build/zephyr/zephyr.dts -- Generated devicetree_generated.h: /workdir/zephyr/build/zephyr/include/generated/devicetree_generated.h -- Including generated dts.cmake file: /workdir/zephyr/build/zephyr/dts.cmake Parsing /workdir/zephyr/samples/net/sockets/echo_server/Kconfig Loaded configuration '/workdir/zephyr/build/zephyr/.config' No change to configuration in '/workdir/zephyr/build/zephyr/.config' No change to Kconfig header in '/workdir/zephyr/build/zephyr/include/generated/autoconf.h' CMake Warning at /workdir/zephyr/subsys/random/CMakeLists.txt:12 (message): Warning: CONFIG_TIMER_RANDOM_GENERATOR is not a truly random generator. This capability is not secure and it is provided for testing purposes only. Use it carefully. CMake Warning at /workdir/zephyr/CMakeLists.txt:853 (message): No SOURCES given to Zephyr library: drivers__entropy Excluding target from build. -- Configuring done -- Generating done -- Build files have been written to: /workdir/zephyr/build -- west build: building application [1/8] Generating misc/generated/syscalls.json, misc/generated/struct_tags.json [2/8] Generating linker_zephyr_pre0.cmd [3/8] Linking C executable zephyr/zephyr_pre0.elf [4/8] Generating linker.cmd [5/8] Generating isr_tables.c [6/8] Building C object zephyr/CMakeFiles/zephyr_final.dir/isr_tables.c.obj [7/8] Linking C executable zephyr/zephyr.elf Memory region Used Size Region Size %age Used FLASH: 148184 B 256 KB 56.53% RAM: 30924 B 32 KB 94.37% IDT_LIST: 0 GB 2 KB 0.00% [8/8] cd /workdir/zephyr/build/zephyr && /usr/bin/python3 /workdir/zephyr/scripts/build/check_init_priorities.py --elf-file=/workdir/zephyr/build/zephyr/zephyr.elf Launching build container using image "docker.io/riot/riotbuild:latest". docker run --rm --tty --user $(id -u) -v '/usr/share/zoneinfo/Europe/Berlin:/etc/localtime:ro' -v '/home/weiss/repos/RIOT-release-manager/RIOT-release:/data/riotbuild/riotbase:delegated' -v '/home/weiss/.cargo/registry:/data/riotbuild/.cargo/registry:delegated' -v '/home/weiss/.cargo/git:/data/riotbuild/.cargo/git:delegated' -e 'RIOTBASE=/data/riotbuild/riotbase' -e 'CCACHE_BASEDIR=/data/riotbuild/riotbase' -e 'BUILD_DIR=/data/riotbuild/riotbase/build' -e 'RIOTPROJECT=/data/riotbuild/riotbase' -e 'RIOTCPU=/data/riotbuild/riotbase/cpu' -e 'RIOTBOARD=/data/riotbuild/riotbase/boards' -e 'RIOTMAKE=/data/riotbuild/riotbase/makefiles' -e 'BOARD=iotlab-m3' -e 'IOTLAB_NODE=m3-10.saclay.iot-lab.info' -e 'DISABLE_MODULE=' -e 'DEFAULT_MODULE=' -e 'FEATURES_REQUIRED=' -e 'FEATURES_BLACKLIST=' -e 'FEATURES_OPTIONAL=' -e 'USEMODULE=auto_init_gnrc_netif auto_init_gnrc_rpl gnrc_icmpv6_echo gnrc_icmpv6_error gnrc_ipv6_router_default gnrc_rpl netdev_default netstats_ipv6 netstats_l2 netstats_rpl ps shell shell_cmd_gnrc_udp shell_cmds_default' -e 'USEPKG=' -w '/data/riotbuild/riotbase/examples/gnrc_networking/' 'docker.io/riot/riotbuild:latest' make 'EXTERNAL_BOARD_DIRS=/data/riotbuild/riotbase/tests/build_system/external_board_dirs/esp-ci-boards' Building application "gnrc_networking" for "iotlab-m3" with MCU "stm32". "make" -C /data/riotbuild/riotbase/pkg/cmsis/ "make" -C /data/riotbuild/riotbase/boards/common/init "make" -C /data/riotbuild/riotbase/boards/iotlab-m3 "make" -C /data/riotbuild/riotbase/boards/common/iotlab "make" -C /data/riotbuild/riotbase/core "make" -C /data/riotbuild/riotbase/core/lib "make" -C /data/riotbuild/riotbase/cpu/stm32 "make" -C /data/riotbuild/riotbase/cpu/cortexm_common "make" -C /data/riotbuild/riotbase/cpu/cortexm_common/periph "make" -C /data/riotbuild/riotbase/cpu/stm32/periph "make" -C /data/riotbuild/riotbase/cpu/stm32/stmclk "make" -C /data/riotbuild/riotbase/cpu/stm32/vectors "make" -C /data/riotbuild/riotbase/drivers "make" -C /data/riotbuild/riotbase/drivers/at86rf2xx "make" -C /data/riotbuild/riotbase/drivers/netdev "make" -C /data/riotbuild/riotbase/drivers/periph_common "make" -C /data/riotbuild/riotbase/sys "make" -C /data/riotbuild/riotbase/sys/auto_init "make" -C /data/riotbuild/riotbase/sys/div "make" -C /data/riotbuild/riotbase/sys/event "make" -C /data/riotbuild/riotbase/sys/evtimer "make" -C /data/riotbuild/riotbase/sys/fmt "make" -C /data/riotbuild/riotbase/sys/frac "make" -C /data/riotbuild/riotbase/sys/isrpipe "make" -C /data/riotbuild/riotbase/sys/libc "make" -C /data/riotbuild/riotbase/sys/luid "make" -C /data/riotbuild/riotbase/sys/malloc_thread_safe "make" -C /data/riotbuild/riotbase/sys/net/crosslayer/inet_csum "make" -C /data/riotbuild/riotbase/sys/net/gnrc "make" -C /data/riotbuild/riotbase/sys/net/gnrc/netapi "make" -C /data/riotbuild/riotbase/sys/net/gnrc/netif "make" -C /data/riotbuild/riotbase/sys/net/gnrc/netif/hdr "make" -C /data/riotbuild/riotbase/sys/net/gnrc/netif/ieee802154 "make" -C /data/riotbuild/riotbase/sys/net/gnrc/netif/init_devs "make" -C /data/riotbuild/riotbase/sys/net/gnrc/netreg "make" -C /data/riotbuild/riotbase/sys/net/gnrc/network_layer/icmpv6 "make" -C /data/riotbuild/riotbase/sys/net/gnrc/network_layer/icmpv6/echo "make" -C /data/riotbuild/riotbase/sys/net/gnrc/network_layer/icmpv6/error "make" -C /data/riotbuild/riotbase/sys/net/gnrc/network_layer/ipv6 "make" -C /data/riotbuild/riotbase/sys/net/gnrc/network_layer/ipv6/hdr "make" -C /data/riotbuild/riotbase/sys/net/gnrc/network_layer/ipv6/nib "make" -C /data/riotbuild/riotbase/sys/net/gnrc/network_layer/ndp "make" -C /data/riotbuild/riotbase/sys/net/gnrc/network_layer/sixlowpan "make" -C /data/riotbuild/riotbase/sys/net/gnrc/network_layer/sixlowpan/ctx "make" -C /data/riotbuild/riotbase/sys/net/gnrc/network_layer/sixlowpan/frag "make" -C /data/riotbuild/riotbase/sys/net/gnrc/network_layer/sixlowpan/frag/fb "make" -C /data/riotbuild/riotbase/sys/net/gnrc/network_layer/sixlowpan/frag/rb "make" -C /data/riotbuild/riotbase/sys/net/gnrc/network_layer/sixlowpan/iphc "make" -C /data/riotbuild/riotbase/sys/net/gnrc/network_layer/sixlowpan/nd "make" -C /data/riotbuild/riotbase/sys/net/gnrc/pkt "make" -C /data/riotbuild/riotbase/sys/net/gnrc/pktbuf "make" -C /data/riotbuild/riotbase/sys/net/gnrc/pktbuf_static "make" -C /data/riotbuild/riotbase/sys/net/gnrc/pktdump "make" -C /data/riotbuild/riotbase/sys/net/gnrc/routing/rpl "make" -C /data/riotbuild/riotbase/sys/net/gnrc/transport_layer/udp "make" -C /data/riotbuild/riotbase/sys/net/link_layer/eui_provider "make" -C /data/riotbuild/riotbase/sys/net/link_layer/ieee802154 "make" -C /data/riotbuild/riotbase/sys/net/link_layer/l2util "make" -C /data/riotbuild/riotbase/sys/net/netif "make" -C /data/riotbuild/riotbase/sys/net/netutils "make" -C /data/riotbuild/riotbase/sys/net/network_layer/icmpv6 "make" -C /data/riotbuild/riotbase/sys/net/network_layer/ipv6/addr "make" -C /data/riotbuild/riotbase/sys/net/network_layer/ipv6/hdr "make" -C /data/riotbuild/riotbase/sys/net/network_layer/sixlowpan "make" -C /data/riotbuild/riotbase/sys/net/transport_layer/udp "make" -C /data/riotbuild/riotbase/sys/newlib_syscalls_default "make" -C /data/riotbuild/riotbase/sys/od "make" -C /data/riotbuild/riotbase/sys/pm_layered "make" -C /data/riotbuild/riotbase/sys/preprocessor "make" -C /data/riotbuild/riotbase/sys/ps "make" -C /data/riotbuild/riotbase/sys/random "make" -C /data/riotbuild/riotbase/sys/shell "make" -C /data/riotbuild/riotbase/sys/shell/cmds "make" -C /data/riotbuild/riotbase/sys/stdio_uart "make" -C /data/riotbuild/riotbase/sys/trickle "make" -C /data/riotbuild/riotbase/sys/tsrb "make" -C /data/riotbuild/riotbase/sys/ztimer text data bss dec hex filename 92196 228 18372 110796 1b0cc /data/riotbuild/riotbase/examples/gnrc_networking/bin/iotlab-m3/gnrc_networking.elf iotlab-node --jmespath='keys(@)[0]' --format='lambda ret: exit(int(ret))' --id 386583 --list saclay,m3,10 --flash /home/weiss/repos/RIOT-release-manager/RIOT-release/examples/gnrc_networking/bin/iotlab-m3/gnrc_networking.bin iotlab-node --jmespath='keys(@)[0]' --format='lambda ret: exit(int(ret))' --id 386583 --list saclay,m3,10 --reset iotlab-node --jmespath='keys(@)[0]' --format='lambda ret: exit(int(ret))' --id 386583 --list saclay,samr21,10 --flash /tmp/zephyrproject/zephyr/build/zephyr/zephyr.elf iotlab-node --jmespath='keys(@)[0]' --format='lambda ret: exit(int(ret))' --id 386583 --list saclay,samr21,10 --reset ssh -t weoss@saclay.iot-lab.info 'socat - tcp:samr21-10.saclay.iot-lab.info:20000' uart:~$ et ipv6 [mnet ipv6 IPv6 support : enabled IPv6 fragmentation support : disabled Multicast Listener Discovery support : enabled Neighbor cache support : enabled Neighbor discovery support : enabled Duplicate address detection (DAD) support : enabled Router advertisement RDNSS option support : enabled 6lo header compression support : enabled Max number of IPv6 network interfaces in the system : 1 Max number of unicast IPv6 addresses per network interface : 3 Max number of multicast IPv6 addresses per network interface : 4 Max number of IPv6 prefixes per network interface : 2 IPv6 addresses for interface 1 (0x20001ec8) (IEEE 802.15.4) ========================================================= Type State Lifetime (sec) Address autoconf preferred infinite fe80::1c90:a3f7:e05d:8e3f/128 manual preferred infinite 2001:db8::1/128 uart:~$ ieee802154 get_pan_id ieee802154 get_pan_id PAN ID 43981 (0xabcd) uart:~$ ieee802154 get_chan ieee802154 get_chan Channel 26 uart:~$ ssh -t weoss@saclay.iot-lab.info 'socat - tcp:m3-10.saclay.iot-lab.info:20000' > ifconfig ifconfig Iface 6 HWaddr: 67:C3 Channel: 26 NID: 0x23 PHY: O-QPSK Long HWaddr: 7A:94:05:FF:25:8D:E7:C3 TX-Power: 0dBm State: IDLE max. Retrans.: 3 CSMA Retries: 4 AUTOACK ACK_REQ CSMA L2-PDU:102 MTU:1280 HL:64 RTR RTR_ADV 6LO IPHC Source address length: 8 Link type: wireless inet6 addr: fe80::7894:5ff:258d:e7c3 scope: link VAL inet6 group: ff02::2 inet6 group: ff02::1 inet6 group: ff02::1:ff8d:e7c3 inet6 group: ff02::1a Statistics for Layer 2 RX packets 1 bytes 43 TX packets 6 (Multicast: 6) bytes 244 TX succeeded 6 errors 0 Statistics for IPv6 RX packets 1 bytes 64 TX packets 6 (Multicast: 6) bytes 370 TX succeeded 6 errors 0 > ifconfig 6 set channel 26 ifconfig 6 set channel 26 success: set channel on interface 6 to 26 > ifconfig 6 set pan_id abcd ifconfig 6 set pan_id abcd success: set network identifier on interface 6 to 0xabcd > udp server start 4242 udp server start 4242 Success: started UDP server on port 4242 > udp send fe80::1c90:a3f7:e05d:8e3f 4242 "RIOT Testing!" udp send fe80::1c90:a3f7:e05d:8e3f 4242 "RIOT Testing!" Success: sent 13 byte(s) to [fe80::1c90:a3f7:e05d:8e3f]:4242 > D: (udp6): UDP (IPv6): Received and replied with 13 bytes PASSED [100%] =============================== warnings summary =============================== 08-interop/test_spec08.py:187 /home/weiss/repos/Release-Specs/08-interop/test_spec08.py:187: DeprecationWarning: invalid escape sequence '\d' match = re.search("PAN ID (?P\d+) \(0x(?P[0-9a-f]+)\)", res) 08-interop/test_spec08.py:193 /home/weiss/repos/Release-Specs/08-interop/test_spec08.py:193: DeprecationWarning: invalid escape sequence '\d' match = re.search("Channel (?P\d+)", res) 08-interop/test_spec08.py::test_task11[nodes0] /home/weiss/repos/Release-Specs/.tox/test/lib/python3.10/site-packages/github/MainClass.py:177: DeprecationWarning: Argument login_or_token is deprecated, please use auth=github.Auth.Token(...) instead warnings.warn( -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html ----- generated xml file: /home/weiss/repos/Release-Specs/test-report.xml ------ ========== 1 passed, 136 deselected, 3 warnings in 101.34s (0:01:41) =========== test: OK (102.09=setup[0.04]+cmd[102.06] seconds) congratulations :) (102.14 seconds) ```
MrKevinWeiss commented 9 months ago

Hmmm. Still many issues with this PR. Currently it would see that the zephyr board crashes on startup now.