search

RIOT-OS / Release-Specs

Specification for RIOT releases and corresponding test configurations
4 stars 21 forks source link

02-tests/task 01 Release 2018.01 - R2 #59

Closed smlng closed 6 years ago

smlng commented 6 years ago

First checkbox marks a tests as run. Second (sub) checkbox indicates success, please add comment on failure.

smlng commented 6 years ago

tests/driver_xbee failes on native as expected, with:

/RIOT/tests/driver_xbee/bin/native/tests_driver_xbee.elf  
RIOT native interrupts/signals initialized.
LED_RED_OFF
LED_GREEN_ON
RIOT native board initialized.
RIOT native hardware initialization complete.

drivers/xbee/xbee.c:478 => 0x2c0c
*** RIOT kernel panic:
FAILED ASSERTION.

*** halted.

native: exiting
smlng commented 6 years ago

tests/mutex_order fails with:

/RIOT/tests/mutex_order/bin/native/tests_mutex_order.elf   
RIOT native interrupts/signals initialized.
LED_RED_OFF
LED_GREEN_ON
RIOT native board initialized.
RIOT native hardware initialization complete.

main(): This is RIOT! (Version: 2018.04-devel-93-gac578-mobi25.inet.haw-hamburg.de-fix_tests_180129)
Mutex order test
Please refer to the README.md for more information

T3 (prio 6): trying to lock mutex now
T4 (prio 4): trying to lock mutex now
T5 (prio 0): trying to lock mutex now
T6 (prio 2): trying to lock mutex now
T7 (prio 1): trying to lock mutex now
T5 (prio 0): locked mutex now
T7 (prio 1): locked mutex now
Unexpected end of file in expect script at "child.expect(u"T\d+ \(prio (\d+)\): locked mutex now")" (tests/mutex_order/tests/01-run.py:28)

Process already stopped
make: *** [test] Error 1
smlng commented 6 years ago

tests/netdev_test fails when DEVELHELP=1 (hence might be expected @miri64 ?) with 

/RIOT/tests/netdev_test/bin/native/tests_netdev_test.elf   
RIOT native interrupts/signals initialized.
LED_RED_OFF
LED_GREEN_ON
RIOT native board initialized.
RIOT native hardware initialization complete.

main(): This is RIOT! (Version: 2018.04-devel-29-g8a60f-mobi1.inet.haw-hamburg.de-HEAD)
sys/net/gnrc/netif/gnrc_netif.c:1157 => 0x804ade1
*** RIOT kernel panic:
FAILED ASSERTION.

*** halted.

native: exiting
Unexpected end of file in expect script at "child.expect_exact('Executing test_get_addr()')" (tests/netdev_test/tests/01-run.py:14)

Process already stopped
Makefile:21: recipe for target 'test' failed
make: *** [test] Error 1

but succeeds with DEVELHELP=0

smlng commented 6 years ago

tests/periph_rtc fails, as it does not fire alarm 4x every 2s as expected

smlng commented 6 years ago

tests/posix_semaphore runs on Linux, but fails on macOS with:

/RIOT/tests/posix_semaphore/bin/native/tests_posix_semaphore.elf   
RIOT native interrupts/signals initialized.
LED_RED_OFF
LED_GREEN_ON
RIOT native board initialized.
RIOT native hardware initialization complete.

main(): This is RIOT! (Version: 2018.04-devel-93-gac578-mobi25.inet.haw-hamburg.de-fix_tests_180129)
######################### TEST1:
first: sem_init
first: thread create
first: thread created
first: sem_getvalue
first: sem_getvalue != 0
first: do yield
second: sem_trywait
second: sem_trywait failed
second: sem_trywait done with == 0
second: wait for post
first: done yield
first: sem_trywait
first: sem_trywait FAILED
first: sem_trywait done
first: sem_post
second: sem was posted
second: end
first: sem_post done
first: sem_destroy
first: end
######################### TEST2:
first: sem_init
first: thread create: 5
first: thread created: priority 5 (1/5)
first: thread create: 4
first: thread created: priority 4 (2/5)
first: thread create: 3
first: thread created: priority 3 (3/5)
first: thread create: 2
first: thread created: priority 2 (4/5)
first: thread create: 1
first: thread created: priority 1 (5/5)
------------------------------------------
post no. 0
Thread 'priority 1' woke up.
Unexpected end of file in expect script at "term.expect_exact("Back in main thread.")" (tests/posix_semaphore/tests/01-run.py:52)

Process already stopped
make: *** [test] Error 1
smlng commented 6 years ago

tests/pthread_rwlock works on Linux, but fails macOS, i.e. exits after second thread finished

smlng commented 6 years ago

tests/rmutex works on Linux, but fails on macOS, i.e. stops/exists after T4

smlng commented 6 years ago

tests/thread_cooperations works on Linux, but fails on macOS, i.e., exists before completion

smlng commented 6 years ago

same for tests/thread_exit

smlng commented 6 years ago

tests/threag_flags works on Linux, but fails on macOS with slightly too high delay

smlng commented 6 years ago

tests/thread_msg_seq works on Linux, but fails on macOS, i.e., exists before completion

smlng commented 6 years ago

tests/xtimer_drift fails on Linux after some time, and on macOS, too. But, on macOS very quickly (<20s) and with higher drift and jitter.

miri64 commented 6 years ago

@smlng can you please open issues in the RIOT repository for your findings? I'm already starting to loose track.

miri64 commented 6 years ago

tests/driver_xbee failes on native as expected,

If you run native on Raspberry Pi you can expose its UART to use xbee.

miri64 commented 6 years ago

tests/netdev_test fails when DEVELHELP=1 (hence might be expected @miri64 ?) with

The whole point of this test is to test the netdev_test framework, so having assertions about the device it might not fulfill is bad, so yes.

miri64 commented 6 years ago

Regarding the other tests: I'm not sure they ever were run on MacOS... But it might be a hint that the RIOT scheduler backend for native on MacOS is broken.

miri64 commented 6 years ago

tests/netdev_test fails when DEVELHELP=1 (hence might be expected @miri64 ?) with

The whole point of this test is to test the netdev_test framework, so having assertions about the device it might not fulfill is bad, so yes.

See https://github.com/RIOT-OS/RIOT/pull/8485

jia200x commented 6 years ago

tests/xtimer_drift fails on Linux after some time, and on macOS, too. But, on macOS very quickly (<20s) and with higher drift and jitter.

Maybe related to RIOT-OS/RIOT#6442

cladmi commented 6 years ago

tests/periph_rtc: fails for good reason, 'rtc_set_time: not implemented' tests/mutex_order: works for me on Linux.

smlng commented 6 years ago

@cladmi yeah saw that too

smlng commented 6 years ago

done