Closed fjmolinas closed 4 years ago
I'm counting as pass since it seems tests/gnrc_tcp
is expected to fail on non-native.
@miri64 I noticed that tests/gnrc_rpl_srh
tends to fail if I do multiple runs, always succeeds after the first flash though. The README
makes me think that is to be expected, but I want to confirm.
IIRC this test was far from stable on boards, even when merged. Is that on native or real board?
Can we just whitelist native then?
If an issue comes with that explaining the reason (I'm not sure there is already one).
I'm counting as pass since it seems
tests/gnrc_tcp
is expected to fail on non-native. 2-tests Task 2 - issues withtests/gnrc_tcp
PASS@miri64 I noticed that
tests/gnrc_rpl_srh
tends to fail if I do multiple runs, always succeeds after the first flash though. TheREADME
makes me think that is to be expected, but I want to confirm.
Yeah... I know by now how to stabilize the sniffer. I'll try to find the time for the next release to fix that.
@MrKevinWeiss ping on xbee
!
@RIOT-OS/maintainers can someone with a 6lowpan rpi test 08-interop
Task 02 and 04.
Re ran tests/gnrc_tcp
on iotlab-m3
, not sure what is going on but now it works reliably...
03-send_data.py: success
06-receive_data_closed_conn.py: success
No protocol specified
Unable to init server: Could not connect: Connection refused
No protocol specified
Unable to init server: Could not connect: Connection refused
(05-garbage-pkts.py:13666): Gdk-CRITICAL **: 11:37:34.476: gdk_cursor_new_for_display: assertion 'GDK_IS_DISPLAY (display)' failed
- test_short_payload SUCCESS
- test_short_header SUCCESS
- test_send_ack_instead_of_syn SUCCESS
- test_option_parsing_term SUCCESS
05-garbage-pkts.py: success
04-receive_data.py: success
02-conn_lifecycle_as_server.py: success
01-conn_lifecycle_as_client.py: success
make: Leaving directory '/home/francisco/workspace/RIOT/tests/gnrc_tcp'
@miri64 what pkt buffer size do you suggest for 03-single-hop-ipv6-icmp Task 6?
03-single-hop-ipv6-icmp Task 3 FAILS in RC1, expected because of https://github.com/RIOT-OS/RIOT/issues/12565
Using CFLAGS+=-DGNRC_PKTBUF_SIZE=12288
The following tests are expected to fail and should be ignored. I'll try to fix failing ones or open issues.
- examples/micropython (missing miniterm on host)
- examples/suit_update (needs `setup_netowrk`)
- tests/gnrc_ipv6_ext (needs sudo)
- tests/gnrc_ipv6_ext_frag (needs sudo)
- tests/gnrc_rpl_srh (needs sudo)
- tests/gnrc_sock_dns (needs sudo)
- tests/gnrc_tcp (needs sudo)
- tests/periph_timer_short_relative_set (fails on most platforms)
- tests/driver_grove_ledbar (invalid config hardfaults some boards)
- tests/driver_my9221 (invalid config hardfaults some boards)
- tests/driver_hd44780 (invalid config hardfaults some boards)
@aabadie regarding the failing tests/pkg_tensorflow-lite
on nucleo-l073rz
and nucleo-f091rc
, they do not seem to have memory issues, shouldn't those arch
actually be blacklisted? because I get a
hard fault, and If I recall correctly you mentioned something about it only working on cortex-m3,m4,m7
2020-01-21 17:58:36,165 # START
2020-01-21 17:58:36,172 # main(): This is RIOT! (Version: 2020.04-devel-2-g9a131-pr_openocd_stm32f7_probe)
2020-01-21 17:58:36,173 # *** RIOT kernel panic:
2020-01-21 17:58:36,173 # HARD FAULT HANDLER
2020-01-21 17:58:36,174 #
2020-01-21 17:58:36,177 # *** rebooting...
2020-01-21 17:58:36,178 #
2020-01-21 17:58:36,189 # Help: Press s to start test, r to print it is ready
^C
regarding the failing tests/pkg_tensorflow-lite on nucleo-l073rz and nucleo-f091rc
This package doesn't work on Cortex-M0 and the problem is unclear to me. Normally they should be added to the BOARD_INSUFFICIENT_MEMORY of the test. Well, it seems that this line is not enough to skip it in compile_and_test_for_board ;)
@smlng could you take a look at the rpi
6lowpan
interop tests? I think you have one.
So far the following tests have failed:
03-single-hop-ipv6-icmp Task #05 - ICMPv6 stress test on native (neighbor cache stress) RIOT-OS/RIOT#12565, but is a know issue.
04-single-hop-6lowpan-icmp Task #10 (Exprimental) - ICMPv6 echo with large payload (IPv6 fragmentation) fixed in RIOT-OS/RIOT#13156
07-multi-hop Task #05 (Experimental) - UDP with large payload on iotlab-m3 with three hops (RPL route) see https://github.com/RIOT-OS/Release-Specs/issues/142 this was already the case in 2019.10 for details see #issuecomment-561677974.
99-run tests Task #01 - Run tests on different hardwares* failures on some boards, but I'll open distinct issues for these, if they get fixed they can get backported but IMO they are not show stoppers.
Tests that were not run:
04-single-hop-6lowpan-icmp Task #07 (Experimental) - ICMPv6 multicast echo with samr21-xpro/zero + xbee, I have not been able to make xbee
work, there might be an issue with our support or wiring on my side. Since this is experimental IMO not a show stopper either, but I will investigate the issue further.
04-single-hop-6lowpan-icmp Task #08 (Experimental) - ICMPv6 echo with samr21-xpro/zero + xbee. Same as above.
01-ci Task #01 - Compile test: @aabadie told me this took 5 days to run last time, maybe he can run again?
- 08-interop Task #02 - ICMPv6 echo between iotlab-m3 and Linux with 6LowPAN, wil run in RC2
- 08-interop Task #04 - ICMPv6 echo between iotlab-m3 and Internet host through Linux with 6LowPAN, will run in RC2
BTW @miri64 do you know if there is an open issue that states why 07-multi-hop Task #05 (Experimental) fails?
Regarding https://github.com/RIOT-OS/RIOT/pull/13182, @kaspar030 @MichelRottleuthner would you suggest re-running all tests because of this?
Other than that IMO only ipv6-frag
related tests need to be re-ran in RC2.
I'll wait for RIOT-OS/RIOT#13182 before creating RC2.
I just finished the tasks involving 6LowPAN and Linux. All OK
Regarding RIOT-OS/RIOT#13182, @kaspar030 @MichelRottleuthner would you suggest re-running all tests because of this?
Don't think that's needed.
* **01-ci** Task #1
Isn't this a relict of when travis used compile_test as main CI script? IMO the test should be reworded as "CI successfully built everything".
Isn't this a relict of when travis used compile_test as main CI script? IMO the test should be reworded as "CI successfully built everything".
I think the difference is that it doesn't use the special compilation flags that the CI uses. There where issues the last time it was built I think @aabadie did it last time, what is your opinion?
I'll go ahead and create RC2 now that RIOT-OS/RIOT#13182 is in.
Closing in favor of #146
Thanks everyone for the help testing RC1 and providing the required bug-fixes, almost everything is GREEN, so RC2 testing should go fast.
This issue lists the status of all tests for the Release Candidate 1 of the 2020.01 release.
Specs tested:
tests/gnrc_tcp
is excludedtests/gnrc_tcp
is excluded