zephyrproject-rtos / zephyr

Primary Git Repository for the Zephyr Project. Zephyr is a new generation, scalable, optimized, secure RTOS for multiple hardware architectures.
https://docs.zephyrproject.org
Apache License 2.0
10.52k stars 6.45k forks source link

Sample posix:eventfd fails on all platforms #26185

Closed PerMac closed 4 years ago

PerMac commented 4 years ago

Describe the bug Sample from ../samples/posix/eventfd fails when run by sanitycheck on-target

To Reproduce Steps to reproduce the behavior: Example with nrf platform, however it fails on every platform (https://testing.zephyrproject.org/daily_tests/zephyr-v2.3.0-327-g636d6cd9cd/report/index.html)

  1. run in zephyr dir ./scripts/sanitycheck --testcase-root ./samples/posix/eventfd --device-testing -p nrf52dk_nrf52832 -v -v --device-serial /dev/ttyACM0
  2. See error

Expected behavior Sample passes.

Impact annoyance

Screenshots or console output


Writing 1 to efd
Writing 2 to efd
Writing 3 to efd
Writing 4 to efd
Completed write loop
About to read
Read lu (0xlx) from efd
Finished
</failure>```

**Environment (please complete the following information):**
 - OS:  Linux
 - Toolchain Zephyr SDK
 - Commit SHA or Version used: zephyr-v2.3.0-327-g636d6cd9cdb6

**Additional context**
The problem is that the sample does not provide a way to verify its correctness.  
PerMac commented 4 years ago

I think the problem can be solved by adding harness: console and regex check in sample.yaml. I've tested it and it seems to work. I can add such regex to the sample.yaml

pfalcon commented 4 years ago

To Reproduce

  1. See error

I don't see an error being quoted in this report.

Locally, with BOARD=frdm_k64f, I see:

[0/1] Flashing frdm_k64f
-- west flash: using runner pyocd
-- runners.pyocd: Flashing file: /home/pfalcon/projects-3rdparty/Embedded/Zephyr/zephyr/sanity-out/frdm_k64f/samples/posix/eventfd/sample.posix.eventfd/zephyr/zephyr.bin
[---|---|---|---|---|---|---|---|---|----]
[========================================]

DEBUG   - DEVICE: About to read
DEBUG   - DEVICE: Read 10 (0xa) from efd
DEBUG   - DEVICE: Finished
DEBUG   - Timed out while monitoring serial output on frdm_k64f
INFO    - 1/1 frdm_k64f                 samples/posix/eventfd/sample.posix.eventfd         FAILED Timeout (device 63.724s)
INFO    - /mnt/hdd/projects-3rdparty/Embedded/Zephyr/zephyr/sanity-out/frdm_k64f/samples/posix/eventfd/sample.posix.eventfd/handler.log
ERROR   - *** Booting Zephyr OS build zephyr-v2.2.0-3558-gcf72fe8fe987  ***

In other words, for some reason, sanitycheck for some reason, without being asked to, reads serial and waits for something, and as it can't get it, it times out, which causes error.

I agree that this needs to be fixed, but:

  1. Why sanitychecks expects something from a sample, without being asked for (there's nothing like that in sample.yaml)?
  2. What it expects for from a random sample? (Again, as it wasn't asked to wait for anythihg in the first place.) Does it expect for some magic word from the sample? How am I, as an an author of a random sample, supposed to know, that there's such a thing as a "sanitycheck", which expects some magic word from my sample? When I write a sample, I'm concerned with giving a good example to users, not to sanitycheck or something. (There're tests to satisfy sanitycheck).
  3. This report is about samples/posix/eventfd, but there're dozens if not hundreds of other samples in there. How do they behave in this regard?

So again, this definitely needs to be fixed, I'm just not sure what and where exactly. If anything, sanitycheck's behavior seems erratic and ungrounded - why suddenly it decided to read and parse output of this sample?

pfalcon commented 4 years ago

So again, this definitely needs to be fixed, I'm just not sure what and where exactly. If anything, sanitycheck's behavior seems erratic and ungrounded - why suddenly it decided to read and parse output of this sample?

In that regard, I submitted https://github.com/zephyrproject-rtos/zephyr/issues/26209, because if the root issue is not fixed, such cases will reoccur again and again.

PerMac commented 4 years ago

I see that introducing the line to be looked for in regex: - "Read 10 (0xa) from efd" causes the sample test failures. It seems that the line is correctly printed out, however, it is not properly detected. Without that line the test passes. Any ideas on how this regex line can be fixed?

PerMac commented 4 years ago

I think I've got it. Adding \ before ( and ) seems to solve the issue

pfalcon commented 4 years ago

Adding \ before ( and ) seems to solve the issue

Indeed, (/) is regex pattern syntax (grouping), not matched literally. Thanks for taking care of this!