riot-ci
commented
2 years ago Closed OlegHahm closed 2 years ago
riot-ci
commented
2 years ago 
OlegHahm
commented
2 years ago @MrKevinWeiss, @benpicco, @fjmolinas, @bergzand, could you help with the interop tests? @kaspar030, could you run the build tests on murdock again?
MrKevinWeiss
commented
2 years ago Sure, I can give it a try tomorrow morning. I thought some were already done by @jia200x. Did you want all (possible) one rerun?
OlegHahm
commented
2 years ago I'm not aware of any previous runs.
MrKevinWeiss
commented
2 years ago MrKevinWeiss
commented
2 years ago Note that the Zephyr node crashes after the echo (we assume it is not riots fault), tested on the old version of zephyr and the new version.
kaspar030
commented
2 years ago Note that the Zephyr node crashes after the echo (we assume it is not riots fault), tested on the old version of zephyr and the new version.
Interop test passed. Other implementation crashed. ;)
aabadie
commented
2 years ago A bit off-topic but the iot-lab team started to port Zephyr to the iotlab-m3. This WIP work is in an iot-lab fork. Basic stuff are working but not the radio... If one day it works, it will be possible to run the interop between 2 iotlab-m3 (1 running RIOT, the other running Zephyr) :smiley:
MrKevinWeiss
commented
2 years ago Just to be clear, also Zephyr board will randomly crash after some time (after about 10 seconds). previous 1 2 3 runs seemed to have varied results and I remember these issues always being present.
@leandrolanzieri had to play around with the buffer sizes to make the firmware fit which may have contributed to it (or not).
Either way I would say "exists in previous releases" or something to that effect.
jia200x
commented
2 years ago All {RIOT, RPI}-border router test passed. Only the ESP tasks are missing. @benpicco you ran some of these tests during last releases, could you give it a try?
jia200x
commented
2 years ago all LoRaWAN test passed. https://github.com/RIOT-OS/Release-Specs/blob/faafe7bcbaf13751b9b07cf5490a7bead13faa27/11-lorawan/README.md#task-06---gnrc-lorawan-abp should be skipped due to https://github.com/RIOT-OS/RIOT/issues/16962
OlegHahm
commented
2 years ago @jia200x, do you know what's up with https://github.com/RIOT-OS/Release-Specs/blob/faafe7bcbaf13751b9b07cf5490a7bead13faa27/11-lorawan/README.md#task-03---abp-join-procedure? It failed. Or is it one of the flaky tests, @fjmolinas mentioned?
OlegHahm
commented
2 years ago I ran 62 random tests on samr21-xpro and got 62 test.success files.
OlegHahm
commented
2 years ago Running the unit-tests on native again with
BUILD_IN_DOCKER=1 RIOT_CI_BUILD=1 dist/tools/compile_and_test_for_board/compile_and_test_for_board.py . native
I got
ERROR:native:Tests failed: 13
Failures during compilation:
- [tests/gnrc_sixlowpan_frag_minfwd](tests/gnrc_sixlowpan_frag_minfwd/compilation.failed)
- [tests/gnrc_sixlowpan_frag_sfr](tests/gnrc_sixlowpan_frag_sfr/compilation.failed)
- [tests/kconfig](tests/kconfig/compilation.failed)
- [tests/kconfig_features](tests/kconfig_features/compilation.failed)
Failures during test:
- [examples/micropython](examples/micropython/test.failed)
- [tests/congure_test](tests/congure_test/test.failed)
- [tests/gnrc_netif_ieee802154](tests/gnrc_netif_ieee802154/test.failed)
- [tests/lwip](tests/lwip/test.failed)
- [tests/netstats_l2](tests/netstats_l2/test.failed)
- [tests/pkg_edhoc_c](tests/pkg_edhoc_c/test.failed)
- [tests/thread_float](tests/thread_float/test.failed)
- [tests/turo](tests/turo/test.failed)
- [tests/ztimer_periodic](tests/ztimer_periodic/test.failed)I thought that RIOT_CI_BUILD=1 should prevent the ztimer_periodic test from being executed!? @kaspar030? What about the other failures? Any idea?
kaspar030
commented
2 years ago I thought that
RIOT_CI_BUILD=1should prevent the ztimer_periodic test from being executed!?
RIOT_CI_BUILD mostly sets options so builds stay reproducible (e.g., not use the git commit hash in RIOT's hello message), and quiets down the build.
Murdock honors TEST_ON_CI_BLACKLIST, I don't think compile_and_test_for_board.py does.
What about the other failures? Any idea?
The compile failures, I don't know.
❯ for i in examples/micropython tests/congure_test tests/gnrc_netif_ieee802154 tests/lwip tests/netstats_l2 tests/pkg_edhoc_c tests/thread_float tests/turo tests/ztimer_periodic ; do grep -H BLACKLIST $i/Makefile; done | grep -E '(all|native)'
tests/lwip/Makefile:TEST_ON_CI_BLACKLIST += all
tests/netstats_l2/Makefile:TEST_ON_CI_BLACKLIST += native
tests/pkg_edhoc_c/Makefile:TEST_ON_CI_BLACKLIST += native
tests/thread_float/Makefile:TEST_ON_CI_BLACKLIST += native
tests/ztimer_periodic/Makefile:TEST_ON_CI_BLACKLIST += nativeInterestingly the compile errors only occur when I build them with BUILD_IN_DOCKER. The reason in all four cases looks similar:
Launching build container using image "riot/riotbuild:latest".
docker run --rm --tty --user $(id -u) -v '/usr/share/zoneinfo/Europe/Berlin:/etc/localtime:ro' -v '/home/oleg/git/RIOT:/data/riotbuild/riotbase:delegated' -v '/home/oleg/.cargo/registry:/data/riotbuild/.cargo/registry:delegated' -v '/home/oleg/.cargo/git:/data/riotbuild/.cargo/git:delegated' -e 'RIOTBASE=/data/riotbuild/riotbase' -e 'CCACHE_BASEDIR=/data/riotbuild/riotbase' -e 'BUILD_DIR=/data/riotbuild/riotbase/build' -e 'RIOTPROJECT=/data/riotbuild/riotbase' -e 'RIOTCPU=/data/riotbuild/riotbase/cpu' -e 'RIOTBOARD=/data/riotbuild/riotbase/boards' -e 'RIOTMAKE=/data/riotbuild/riotbase/makefiles' -e 'DISABLE_MODULE=' -e 'DEFAULT_MODULE=test_utils_interactive_sync test_utils_print_stack_usage' -e 'FEATURES_REQUIRED=' -e 'FEATURES_BLACKLIST=' -e 'FEATURES_OPTIONAL=' -e 'USEMODULE=external_module_1 external_module_2' -e 'USEPKG=external_pkg_1 external_pkg_2' -e 'KCONFIG_ADD_CONFIG=' -w '/data/riotbuild/riotbase/tests/kconfig/' 'riot/riotbuild:latest' make 'EXTERNAL_MODULE_DIRS=/data/riotbuild/riotbase/tests/kconfig/external_modules'
Building application "tests_kconfig" for "native" with MCU "native".
"make" -C /data/riotbuild/riotbase/tests/kconfig/external_pkgs/external_pkg_1/
"make" -C /data/riotbuild/riotbase/tests/kconfig/external_pkgs/external_pkg_2/
/data/riotbuild/riotbase/dist/tools/fixdep/fixdep: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /data/riotbuild/riotbase/dist/tools/fixdep/fixdep)
/data/riotbuild/riotbase/dist/tools/fixdep/fixdep: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /data/riotbuild/riotbase/dist/tools/fixdep/fixdep)
make[1]: *** [/data/riotbuild/riotbase/Makefile.base:147: /data/riotbuild/riotbase/tests/kconfig/bin/native/application_tests_kconfig/main.o] Error 1
make: *** [/data/riotbuild/riotbase/Makefile.include:737: application_tests_kconfig.module] Error 2
make: *** [/home/oleg/git/RIOT/makefiles/docker.inc.mk:350: ..in-docker-container] Error 2The tests build just fine when built without docker.
OlegHahm
commented
2 years ago Okay, two other tests failed because python-rapidjson was not installed on my machine.
So, basically all tests that should succeed now, except for tests/gnrc_netif_ieee802154 which fails with:
% make test [22:31:13][0]
r
/home/oleg/git/RIOT/tests/gnrc_netif_ieee802154/bin/native/tests_gnrc_netif_ieee802154.elf -z "0.0.0.0:17755,localhost:17754" /dev/ttyACM0
RIOT native interrupts/signals initialized.
RIOT native board initialized.
RIOT native hardware initialization complete.
/home/oleg/git/RIOT/tests/gnrc_netif_ieee802154/bin/native/tests_gnrc_netif_ieee802154.elf: ZEP: Unable to connect socket: Bad file descriptor@miri64, do you have an idea here?
kaspar030
commented
2 years ago So, basically all tests that should succeed now, except for tests/gnrc_netif_ieee802154 which fails with:
The python tests script uses "scapy", maybe that needs to be installed?
benpicco
commented
2 years ago 2022-04-27 23:20:17,518 # rst:0x7 (TG0WDT_SYS_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
2022-04-27 23:20:17,520 # configsip: 0, SPIWP:0xee
2022-04-27 23:20:17,526 # clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
2022-04-27 23:20:17,528 # mode:DOUT, clock div:2
2022-04-27 23:20:17,530 # load:0x3fff0018,len:4
2022-04-27 23:20:17,533 # load:0x3fff001c,len:4004
2022-04-27 23:20:17,535 # load:0x40078000,len:7304
2022-04-27 23:20:17,537 # load:0x40080000,len:7224
2022-04-27 23:20:17,539 # entry 0x40080350
2022-04-27 23:20:17,545 # W (29) boot: PRO CPU has been reset by WDT.
2022-04-27 23:20:17,549 # W (29) boot: WDT reset info: PRO CPU PC=0x400803c3
2022-04-27 23:20:17,554 # W (29) boot: WDT reset info: APP CPU PC=0x2f3378fb
2022-04-27 23:20:17,785 #
2022-04-27 23:20:19,525 # NETOPT_RX_END_IRQ not implemented by driver
2022-04-27 23:20:19,790 # Serial port disconnected, waiting to get reconnected...
2022-04-27 23:20:20,792 # Try to reconnect to /dev/ttyUSB1 again...
2022-04-27 23:20:20,799 # Reconnected to serial port /dev/ttyUSB1
2022-04-27 23:20:25,524 # ets Jun 8 2016 00:22:57
2022-04-27 23:20:25,525 #
2022-04-27 23:20:25,530 # rst:0x7 (TG0WDT_SYS_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
2022-04-27 23:20:25,532 # configsip: 0, SPIWP:0xee
2022-04-27 23:20:25,538 # clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
2022-04-27 23:20:25,540 # mode:DOUT, clock div:2
2022-04-27 23:20:25,542 # load:0x3fff0018,len:4
2022-04-27 23:20:25,544 # load:0x3fff001c,len:4004
2022-04-27 23:20:25,547 # load:0x40078000,len:7304
2022-04-27 23:20:25,549 # load:0x40080000,len:7224
2022-04-27 23:20:25,551 # entry 0x40080350
2022-04-27 23:20:25,557 # W (29) boot: PRO CPU has been reset by WDT.
2022-04-27 23:20:25,561 # W (29) boot: WDT reset info: PRO CPU PC=0x4008e5ac
2022-04-27 23:20:25,566 # W (29) boot: WDT reset info: APP CPU PC=0x2f3378fb
2022-04-27 23:20:25,796 #
2022-04-27 23:20:27,537 # NETOPT_RX_END_IRQ not implemented by driverboard is stuck in a boot loop
2022-04-27 23:18:34,298 # rst:0x3 (SW_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
2022-04-27 23:18:34,300 # configsip: 0, SPIWP:0xee
2022-04-27 23:18:34,307 # clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
2022-04-27 23:18:34,309 # mode:DOUT, clock div:2
2022-04-27 23:18:34,311 # load:0x3fff0018,len:4
2022-04-27 23:18:34,313 # load:0x3fff001c,len:4004
2022-04-27 23:18:34,315 # load:0x40078000,len:7304
2022-04-27 23:18:34,317 # load:0x40080000,len:7224
2022-04-27 23:18:34,319 # entry 0x40080350
2022-04-27 23:18:34,562 #
2022-04-27 23:18:36,302 # NETOPT_RX_END_IRQ not implemented by driver
2022-04-27 23:18:36,466 # phy_version: 3910, c0c45a3, May 21 2018, 18:07:06, 0, 2
2022-04-27 23:18:36,475 # NETOPT_RX_END_IRQ not implemented by driver
2022-04-27 23:18:36,479 # DHCPv6: Selecting interface 9 as upstream
2022-04-27 23:18:36,482 # DHCPv6: Selecting interface 9 as upstream
2022-04-27 23:18:36,488 # main(): This is RIOT! (Version: 2022.04-devel-5-g6a8ded-2022.01-branch)
2022-04-27 23:18:36,492 # RIOT border router example application
2022-04-27 23:18:36,494 # All up, running the shell now
2022-04-27 23:18:48,203 # ping riot-os.org
2022-04-27 23:18:48,239 # 12 bytes from 2001:67c:254:b0b0::1: icmp_seq=0 ttl=60 time=20.864 ms
2022-04-27 23:18:49,239 # 12 bytes from 2001:67c:254:b0b0::1: icmp_seq=1 ttl=60 time=21.034 ms
2022-04-27 23:18:50,239 # 12 bytes from 2001:67c:254:b0b0::1: icmp_seq=2 ttl=60 time=20.998 ms
2022-04-27 23:18:50,239 #
2022-04-27 23:18:50,242 # --- riot-os.org PING statistics ---
2022-04-27 23:18:50,247 # 3 packets transmitted, 3 packets received, 0% packet loss
2022-04-27 23:18:50,251 # round-trip min/avg/max = 20.864/20.965/21.034 msThis is with esp32-wroom-32 and xtensa-esp32-elf-gcc (crosstool-NG crosstool-ng-1.22.0-80-g6c4433a5) 5.2.0, same result with xtensa-esp32-elf-gcc (crosstool-NG esp-2021r2-patch3) 8.4.0
jia200x
commented
2 years ago @jia200x, do you know what's up with https://github.com/RIOT-OS/Release-Specs/blob/faafe7bcbaf13751b9b07cf5490a7bead13faa27/11-lorawan/README.md#task-03---abp-join-procedure? It failed. Or is it one of the flaky tests, @fjmolinas mentioned?
I ran the semtech lorawan ABP test manually and passed
OlegHahm
commented
2 years ago So, basically all tests that should succeed now, except for tests/gnrc_netif_ieee802154 which fails with:
The python tests script uses "scapy", maybe that needs to be installed?
Hm, scapy was installed already. Does the test work on your machine?
kaspar030
commented
2 years ago Hm, scapy was installed already. Does the test work on your machine?
Yes, tests/gnrc_netif_ieee802154 on native passes here!
OlegHahm
commented
2 years ago So, networking on esp32 is apparently broken!? I tend to call this a blocker for the release.
OlegHahm
commented
2 years ago Hm, scapy was installed already. Does the test work on your machine?
Yes,
tests/gnrc_netif_ieee802154on native passes here!
Good enough for me. :)
benpicco
commented
2 years ago Found the issue on esp32:
08196603030b735dff4d1569b850ea720d6d5c38 is the first bad commit commit 08196603030b735dff4d1569b850ea720d6d5c38 Author: @fjmolinas Date: Wed Mar 2 10:00:00 2022 +0100
sys/syscalls: add libc_gettimeofday
Conditionally implement gettimeofday() if module is included, this
avoids including ztimer64 even when not neededcpu/native/Kconfig | 1 - cpu/native/Makefile.dep | 9 ++++++--- cpu/native/syscalls.c | 5 +++-- makefiles/pseudomodules.inc.mk | 4 ++++ pkg/lua/Makefile.dep | 8 ++++++++ pkg/tinydtls/Makefile.dep | 8 ++++++++ pkg/wolfssl/Makefile.dep | 8 ++++++++ sys/Kconfig | 8 +++++++- sys/Kconfig.newlib | 9 ++++++++- sys/Makefile.dep | 9 ++++++--- sys/newlib_syscalls_default/syscalls.c | 14 +++----------- tests/cpp11_mutex/Makefile | 1 + 12 files changed, 62 insertions(+), 22 deletions(-)
With that commit reverted:
2022-04-27 23:42:57,362 # NETOPT_RX_END_IRQ not implemented by driver
2022-04-27 23:42:57,527 # phy_version: 3910, c0c45a3, May 21 2018, 18:07:06, 0, 2
2022-04-27 23:42:57,537 # NETOPT_RX_END_IRQ not implemented by driver
2022-04-27 23:42:57,540 # DHCPv6: Selecting interface 9 as upstream
2022-04-27 23:42:57,544 # DHCPv6: Selecting interface 9 as upstream
2022-04-27 23:42:57,549 # main(): This is RIOT! (Version: 2022.07-devel-256-g9538b8)
2022-04-27 23:42:57,552 # RIOT border router example application
2022-04-27 23:42:57,555 # All up, running the shell now
2022-04-27 23:43:06,700 # > ping riot-os.org
2022-04-27 23:43:07,735 # 12 bytes from 2001:67c:254:b0b0::1: icmp_seq=1 ttl=60 time=21.459 ms
2022-04-27 23:43:08,744 # 12 bytes from 2001:67c:254:b0b0::1: icmp_seq=2 ttl=60 time=31.146 ms
2022-04-27 23:43:09,707 #
2022-04-27 23:43:09,710 # --- riot-os.org PING statistics ---
2022-04-27 23:43:09,715 # 3 packets transmitted, 2 packets received, 33% packet loss
2022-04-27 23:43:09,720 # round-trip min/avg/max = 21.459/26.302/31.146 ms
miri64
commented
2 years ago @miri64, do you have an idea here?
Maybe something went wrong in the socket initialization? But not sure. Happy that it seem to have resolved itself :D
OlegHahm
commented
2 years ago Closing in favor of #248
This issue lists the status of all tests for the Release Candidate 6 of the 2022.04 release.
Specs tested:
Task #09 - deprecatedTask #10 - deprecated