Intermittent test failure in 11-obsolete?

[nabijaczleweli]

I get this a stochastic half the time when running make check repeatedly on my laptop:

     Running tests/test_cases.rs (target/release/deps/test_cases-dba1a1b0414c78cc)

running 12 tests
test test_05_kernel_disabled ... ok
test test_03_too_much_memory ... ok
test test_01_basic ... ok
test test_07a_mount_point_excl ... ok
test test_06_kernel_enabled ... ok
test test_04_dropins ... ok
test test_07_mount_point ... ok
test test_02_zstd ... ok
test test_08_plain_device ... ok
test test_09_zram_size ... ok
test test_10_example ... ok
test test_11_obsolete ... FAILED

failures:

---- test_11_obsolete stdout ----
stdout:Only in tests/11-obsolete/run.expected/units: dev-zram1.swap
Only in tests/11-obsolete/run.expected/units/swap.target.wants: dev-zram1.swap
Only in tests/11-obsolete/run.expected/units: systemd-zram-setup@zram1.service.d

thread 'test_11_obsolete' panicked at 'assertion failed: diff.status.success()', tests/test_cases.rs:93:5
stack backtrace:
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

failures:
    test_11_obsolete

test result: FAILED. 11 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.07s

error: test failed, to rerun pass '--test test_cases'
make: *** [Makefile:41: check] Error 101

Not a clue why since it works the other half?

[keszybz]

No idea why it would fail. Does this also happen when running just that test, e.g. as target/debug/deps/test_cases-44059338f2a6f9e4 test_11_obsolete ?

[nabijaczleweli]

It does! Here's strace -f: ss-ok.gz ss-fail.gz

Oddly, we do indeed seem to not be generating the files in the failing case: As to why – not a clue yet

[nabijaczleweli]

Log says that uh, in the failing case we're eliding zram1 due to host-memory-limit?

[nabijaczleweli]

Ah, here we are! The ordering of the ini K/Vs is stochastic (in a HashMap, seeded by getrandom(2)), but the ordering in 11/e/s/z-g.c assumes in-order reads:

Or, well, actually, it isn't anymore! On my laptop I still have the Cargo.lock from when I first started doing z-g in 2018 with rust-ini 0.13, which just used a HashMap. 0.14 uses indexmap for ordered keys behind an optional feature flag. 0.15 is the first to use ordered-multimap, and always does so.

After bumping the minimum to 0.15, this test now passed 1000 consecutive runs on both 0.15 and 0.17.