search

rnpgp / rnp

RNP: high performance C++ OpenPGP library used by Mozilla Thunderbird
https://www.rnpgp.org
Other
194 stars 55 forks source link

intermittent test suite failures when run in parallel #1408

Closed dkg closed 3 years ago

dkg commented 3 years ago

When i run the test suite for v0.14.0 in parallel on a machine with many cores, i get intermittent failures.

In particular, it looks like cli_tests.py is run multiple times concurrently, and sometimes two runs will try to produce the same output file at the same time. During a failing test, i see this sort of output/warning fly by:

File '/tmp/x/rnp/build/src/tests/cleartext.rnp' already exists. Would you like to overwrite it (y/N)? Generating a new key...

I want to be able to run the test suite in parallel -- it's a large and comprehensive suite. But i'm forced to run it serially to avoid these intermittent failures.

dewyatt commented 3 years ago

This is a (probably undocumented) bug that occurs when the rnp build directory is in /tmp.

dkg commented 3 years ago

i can confirm that i've only seen these failures when the build directory is in /tmp, but i'm mystified as to why that should be the case. can you explain more?

dewyatt commented 3 years ago

i can confirm that i've only seen these failures when the build directory is in /tmp, but i'm mystified as to why that should be the case. can you explain more?

It's likely to be from: https://github.com/rnpgp/rnp/blob/c04acfda7b5faf6bec2960cb8156fd93362411b0/src/tests/cli_tests.py#L751-L753

dkg commented 3 years ago

thanks for the pointer. #1420 appears to resolve the issue for me.