Closed MrKevinWeiss closed 5 years ago
6% packet loss
High packet loss
It seems like the high packet lost could be because of the async ping6
It seems like the high packet lost could be because of the async ping6
Looks like, so the test parameters should be adopted.
I am working on that and will give a PR to update
I think it is fine with -i 350
Running the build test on my cpu seems to work on everything but the following boards, likely due to not having the toolchain. Though not everything is there since my console truncated the total output (I should have piped to a file). Since everything works in docker I am inclined to say it isn't a problem. I may try to run over the weekend again after installing all the toolchains.
arduino-leonardo
chronos
hifive1
mips-malta
msb-430
msb-430h
pic32-clicker
pic32-wifire
telosb
wsn430-v1_3b
wsn430-v1_4
z1
Errors:
Outcome:
success: riotboot, ccn-lite-relay, gnrc_networking_mac, lorawan, lua_REPL, lua_basic, nimble_gatt, nimble_heart_rate_sensor, nimble_scanner, openthread, skald_eddystone, skald_ibeacon, usbus_minimal, board_calliope-mini, board_microbit, build_system_utils, conn_can, cortexm_common_ldscript, cpu_cortexm_address_check, cpu_efm32_features, driver_at30tse75x, driver_bmx055, driver_ds18, driver_motor_driver, driver_mpu9150, driver_srf02, driver_tsl4531x, emb6, gnrc_gomach, gnrc_lwmac, lua_loader, mpu_stack_guard, netstats_l2, nimble_l2cap, nimble_l2cap_server, periph_dac, periph_dma, periph_eeprom, periph_i2c, periph_pwm, periph_qdec, periph_uart_mode, pkg_cmsis-dsp, pkg_fatfs_vfs, pkg_oonf_api, pkg_semtech-loramac, pkg_tinycrypt, pkg_ubasic, puf_sram, riotboot, riotboot_flashwrite, socket_zep, trace, usbus_cdc_ecm, warn_conflict
failed: arduino_hello-world, asymcute_mqttsn, bindist, cord_ep, cord_epsim, default, dtls-echo, emcute_mqttsn, filesystem, gcoap, gnrc_border_router, gnrc_minimal, gnrc_networking, gnrc_tftp, hello-world, ipc_pingpong, javascript, nanocoap_server, ndn-ping, posix_sockets, riot_and_cpp, saul, timer_periodic_wakeup, bench_msg_pingpong, bench_mutex_pingpong, bench_runtime_coreapis, bench_sched_nop, bench_sizeof_coretypes, bench_thread_flags_pingpong, bench_thread_yield_pingpong, bench_timers, bitarithm_timings, bloom_bytes, buttons, can_trx, cb_mux, cb_mux_bench, cond_order, cpp11_condition_variable, cpp11_mutex, cpp11_thread, driver_ad7746, driver_adcxx1c, driver_ads101x, driver_adt7310, driver_adxl345, driver_apa102, driver_at, driver_at86rf2xx, driver_ata8520e, driver_bh1750, driver_bmp180, driver_bmx280, driver_ccs811, driver_ccs811_full, driver_dht, driver_ds1307, driver_ds3234, driver_ds75lx, driver_dsp0401, driver_dynamixel, driver_enc28j60, driver_encx24j600, driver_feetech, driver_fxos8700, driver_grove_ledbar, driver_hd44780, driver_hdc1000, driver_hih6130, driver_hts221, driver_ina220, driver_io1_xplained, driver_isl29020, driver_isl29125, driver_jc42, driver_kw2xrf, driver_l3g4200d, driver_lc709203f, driver_lis2dh12, driver_lis3dh, driver_lis3mdl, driver_lpd8808, driver_lpsxxx, driver_lsm303dlhc, driver_lsm6dsl, driver_ltc4150, driver_mag3110, driver_mma7660, driver_mma8x5x, driver_mpl3115a2, driver_mq3, driver_my9221, driver_nrf24l01p_lowlevel, driver_nvram_spi, driver_pcd8544, driver_pir, driver_pn532, driver_pulse_counter, driver_rn2xx3, driver_sdcard_spi, driver_sds011, driver_servo, driver_sht1x, driver_sht2x, driver_sht3x, driver_si114x, driver_si70xx, driver_soft_spi, driver_srf04, driver_srf08, driver_sx127x, driver_tcs37727, driver_tmp006, driver_tps6274x, driver_tsl2561, driver_vcnl40x0, driver_veml6070, driver_xbee, eepreg, embunit, event_wait_timeout, events, evtimer_msg, evtimer_underflow, external_module_dirs, fault_handler, float, fmt_print, gnrc_ipv6_ext, gnrc_ipv6_fwd_w_sub, gnrc_ipv6_nib, gnrc_ipv6_nib_6ln, gnrc_mac_timeout, gnrc_ndp, gnrc_netif, gnrc_rpl_srh, gnrc_sixlowpan, gnrc_sixlowpan_frag, gnrc_sock_dns, gnrc_sock_ip, gnrc_sock_udp, gnrc_tcp_client, gnrc_tcp_server, gnrc_udp, irq, isr_yield_higher, l2util, leds, libc_newlib, libfixmath, libfixmath_unittests, lwip, lwip_sock_ip, lwip_sock_tcp, lwip_sock_udp, malloc, mcuboot, memarray, minimal, msg_avail, msg_send_receive, msg_try_receive, mutex_order, mutex_unlock_and_sleep, nanocoap_cli, netdev_test, nhdp, od, periph_adc, periph_cpuid, periph_flashpage, periph_gpio, periph_gpio_arduino, periph_hwrng, periph_pm, periph_rtc, periph_rtt, periph_spi, periph_timer, periph_uart, pipe, pkg_c25519, pkg_cayenne-lpp, pkg_cn-cbor, pkg_fatfs, pkg_hacl, pkg_heatshrink, pkg_jsmn, pkg_libb2, pkg_libcoap, pkg_libcose, pkg_libhydrogen, pkg_littlefs, pkg_lora-serialization, pkg_micro-ecc, pkg_micro-ecc-with-hwrng, pkg_microcoap, pkg_minmea, pkg_monocypher, pkg_nanocbor, pkg_qdsa, pkg_relic, pkg_spiffs, pkg_tiny-asn1, pkg_tinycbor, pkg_tweetnacl, pkg_u8g2, pkg_ucglib, pkg_umorse, posix_semaphore, posix_time, ps_schedstatistics, pthread, pthread_barrier, pthread_cleanup, pthread_condition_variable, pthread_cooperation, pthread_rwlock, pthread_tls, riotboot_hdr, rmutex, rng, saul, sched_testing, shell, slip, sntp, ssp, stdin, struct_tm_utility, sys_arduino, thread_basic, thread_cooperation, thread_exit, thread_flags, thread_flags_xtimer, thread_float, thread_flood, thread_msg, thread_msg_block_race, thread_msg_block_w_queue, thread_msg_block_wo_queue, thread_msg_seq, thread_priority_inversion, thread_race, trickle, unittests, xtimer_drift, xtimer_hang, xtimer_longterm, xtimer_msg, xtimer_msg_receive_timeout, xtimer_mutex_lock_timeout, xtimer_now64_continuity, xtimer_periodic_wakeup, xtimer_remove, xtimer_reset, xtimer_usleep, xtimer_usleep_short
Errors: 0 Warnings: 0
I added an issue for the fact I found during 01-ci - Task #01
that buildtest
hides some failures https://github.com/RIOT-OS/RIOT/issues/11842
I will try a fix.
Other wise, I could correctly compile with BUILD_IN_DOCKER=1
for both gnu and all boards
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin BUILD_IN_DOCKER=1 ./task01.py /home/harter/work/git/worktree/riot_release
For TOOLCHAIN=llvm
, I successfully tested the iotlab-m3 native nrf52dk mulle nucleo-f401re samr21-xpro slstk3402a
as done by murdock
. I needed to change the boards list in the file, as providing it through environment variable to docker was not handled. Must be a space
handling issue.
It seems like 03-single-hop-ipv6-icmp Task 04 has some issues but I don't know what I should be looking for and how making the ping async affects everything.
I was pinging one node with 9 other instances of native and started getting some odd messages. I could use some insight if this is expected or if the test needs to be adapted.
Finally the node that I was pinging crashed... I think this is a bug, @miri64 can you look into it?
@cgundogan might also be able to help?
It seems like -DGNRC_IPV6_NIB_NUMOF=10
helps the situation
I still get some packet loss but it isn't throwing errors
Task 11.01 fails for me.
I can see messages in TTN, and it joins without any issues. I will investigate further
In fact, there's somethying wrong with the loramac shell commands:
> loramac set appskey 1EBB10A1AE58065BB4540A4181DC7BB8
2019-07-15 17:16:52,784 - INFO # loramac set appskey 1EBB10A1AE58065BB4540A4181DC7BB8
> loramac get appskey
2019-07-15 17:16:55,244 - INFO # loramac get appskey
2019-07-15 17:16:55,248 - INFO # APPSKEY: E9030008000000000000000000000000
EDIT: Both set nwkskey
and set appskey
seem to be broken...
EDIT: Both set nwkskey and set appskey seem to be broken...
@jia200x are you testing on iotlab?
@jia200x are you testing on iotlab?
Nope, locally on a b-l072z-lrwan1
. For some reason, I cannot set the keys properly. I'm investigating why.
Success:
unittests
on iotlab-m3
The unittests
do not compile on boards without periph_hwrng
anymore…
Yes "unittests" that have "hardware requirements"…
This is bad
This added the periph_hwrng
requirement for tests/unittests
https://github.com/RIOT-OS/RIOT/pull/7421
Nope, locally on a b-l072z-lrwan1. For some reason, I cannot set the keys properly. I'm investigating why.
I'm getting the same result as you.
Maybe it's time to review (and merge ;) ) https://github.com/RIOT-OS/RIOT/pull/11783.
Cheers!
Maybe it's time to review (and merge ;) ) RIOT-OS/RIOT#11783.
Cheers!
Would still be a good idea to know what broke it, if it is indeed the same as https://github.com/RIOT-OS/RIOT/issues/11626 or not.
Would still be a good idea to know what broke it, if it is indeed the same as RIOT-OS/RIOT#11626 or not.
Seems to be a related problem, the problem with the nwskey was introduced by RIOT-OS/RIOT#11626, just after that PR it is working, and broken after wards. I'm looking into it too.
RIOT-OS/RIOT#11783 does fix the issue. I will try to understand the problem better and see if I can get a formal explanation. But as @aabadie pointed out in the PR it is not the nicest solution but it works, back-porting RIOT-OS/RIOT#11783 could be the sensible solution.
@cladmi so it seems like the problem with the unittest failing is only due to incorrect test placement. Does this need to be backported or just mentioned in the release notes and an issue raised to move it out of unittests and into its own test as @miri64 suggested in the PR?
I opened RIOT-OS/RIOT#11847 that fixes the issue with the lorawan tasks, its an alternative to RIOT-OS/RIOT#11783.
I am running Task03.4 in the background to try to reproduce your assertion error @MrKevinWeiss.
For Task3.5 using many native node to ping one node I have recurring pktbuf hex dumps that do not get cleared.
sudo ./dist/tools/tapsetup/tapsetup -c 11
tmux new-session -d -s nodes "make flash term -C tests/gnrc_udp"
for i in {1..10}; do tmux new-window -t nodes: -n $i "make term PORT=tap$i -C tests/gnrc_udp/"; done
tmux attach -t nodes:0
ifconfig
command and copy the ctrl+b d
for i in {1..10}; do tmux send-keys -t nodes:$i "ping6 -c 1000000 -i 0 -s 1452 <ll_addr>" enter; done
for i in {0..10}; do tmux send-keys -t nodes:$i "pktbuf" enter; done
tmux attach
and scroll with ctrl+b n
for i in {1..10}; do tmux send-keys -t nodes:$i "ping6 -c 100000 -i 1 -s 1452 <ll_addr>" enter; done
I still don't know if native should be able to handle this or not. It seems pretty heavy.
Yepp it should be :-/. I'll have a look if this was also the case in 2019.04 or if this is a regression.
Should I open a proper issue?
@cladmi so it seems like the problem with the unittest failing is only due to incorrect test placement. Does this need to be backported or just mentioned in the release notes and an issue raised to move it out of unittests and into its own test as @miri64 suggested in the PR?
It is step 1.3 of the release spec.
Let me first try to reproduce.
Up until 11. I can not reproduce your finding on 2019.04-RC1. Still waiting for 12-13.
Up until 11. I can not reproduce your finding on 2019.04-RC1. Still waiting for 12-13.
12-13 I also wasn't able to reproduce on 2019.04-RC1. I will focus on 1-11 to find the regression.
Up until 11. I can not reproduce your finding on 2019.04-RC1. Still waiting for 12-13.
12-13 I also wasn't able to reproduce on 2019.04-RC1. I will focus on 1-11 to find the regression.
Ran my script about 10 times and was not able to reproduce on 2019.07-RC1 :-/.
Updated a bug in the script. Still can't reproduce.
From what @MrKevinWeiss and I were able to assess offline it looks like some rare race-condition on native, that might not happen on all system is the cause... The packets that remain in the packet buffer look like they should after leaving the ICMPv6 module here and still have the gnrc_netif_hdr_t
they got in that function, so they did not pass by here yet. The only way this can happen if they are not properly released after an unsuccessful dispatch (I checked, they are) or if they somehow got stuck in gnrc_ipv6
's message queue. I try to reproduce it on a different system (or maybe someone else with a Lenovo X1 Carbon can try). If I'm not able to we should just mark it as a known issue and document the steps to reproduce in an issue upstream.
On my laptop I do get the gnrc_netif: possibly lost interrupt.
which I don't get on my work machine. I did not however see the filled packet buffer, even after several retries and it never froze. I did see similar packets to what @MrKevinWeiss sent me offline hanging out in the packet buffer for a while though but they later disappeared (they probably were stuck in the neighbor cache's queue where a new netif
header is also added).
I split the test outside https://github.com/RIOT-OS/RIOT/pull/11855
I change that buildtest now behaves like doing make all
for each board.
https://github.com/RIOT-OS/RIOT/pull/11857
This issue lists the status of all tests for the Release Candidate 1 of the 2019.07 release.
Specs tested: