rust-lang / libc

Raw bindings to platform APIs for Rust
https://docs.rs/libc
Apache License 2.0
2.07k stars 1.04k forks source link

A heisenbug in aarch64-linux-android CI #2081

Open workingjubilee opened 3 years ago

workingjubilee commented 3 years ago

CI for that target is currently disabled due to this issue.

Hello, traveler.

I have stories to tell of strange bugs from unseen lands. Monsters which flicker in and out of sight, vanishing in the blink of an eye, hidden in virtual domains that themselves coalesce into being and then topple back into sand.

This primarily affects the aarch64-linux-android CI but it also affects the sparc-unknown-linux-gnu CI. This was found at least as far back at #1851. Several error logs are quoted into #2078. The associated errors seemed to occur more often during the middle of the day rather than at night, so it may be an occurrence of a "noisy neighbor" problem in the virtual environment. It may also be an issue with QEMU.

This happens near the the cmsg tests, but can also happen during errqueue tests, the primary messages are

cmsg

```backtrace stderr --- thread 'main' panicked at 'No option 'h' defined', /cargo/registry/src/github.com-1ecc6299db9ec823/getopts-0.2.21/src/lib.rs:799:21 note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace thread 'main' panicked at 'failed to find successful test run', /tmp/runtest.rs:45:9 note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace error: test failed, to rerun pass '--test cmsg' ```

errqueue

```backtrace stderr --- thread 'main' panicked at 'No option 'h' defined', /cargo/registry/src/github.com-1ecc6299db9ec823/getopts-0.2.21/src/lib.rs:799:21 stack backtrace: note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. thread 'main' panicked at 'failed to find successful test run', /tmp/runtest.rs:46:9 stack backtrace: 0: std::panicking::begin_panic 1: runtest::main::{{closure}} 2: core::option::Option::unwrap_or_else 3: runtest::main 4: core::ops::function::FnOnce::call_once note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. error: test failed, to rerun pass '--test errqueue' ```

In 5c07fcfd1c1333c5e6632ec96e169bd832ac5b57 we decided to add, and for the moment keep, RUST_BACKTRACE=1 at appropriate points in CI so as to hopefully surface a more useful backtrace when this triggers again. In fcae5a7a1a65b56d89f38cafdc59659d1f756620 we decided to cut aarch64-linux-android out of CI.

But we could do no more than place that warding to blockade that cyclopean archway. The monster still lives, and in time, it may come again, between the blinks of our strange existences.

maxbla commented 3 years ago

Link to that line in getopts. I'm not at all familiar with this crate's tests and looked through, but can't find which CI tests depend on getopts.