search

pantoniou / libfyaml

Fully feature complete YAML parser and emitter, supporting the latest YAML spec and passing the full YAML testsuite.
MIT License
243 stars 74 forks source link

Parallel test execution breaks tests due to unfinished git clone #72

Closed schroeding closed 1 year ago

schroeding commented 1 year ago

When running the test suite in parallel, the two tests testsuite.test and jsontestsuite.test may be executed before their dependencies test-suite-data and json-test-suite-data are finished, as the download / cloning of the git repository takes a considerable amount of time. In this case, those two tests fail due to missing test data.

You may be able to reproduce this issue by running make check with the -j flag, e.g. env TESTS="jsontestsuite.test" make -e -j16 check. If the test is executed too fast / before the git clone(s) of test-suite-data and json-test-suite-data are finished, the test log file will include errors like:

[ERR]: failed to open json-test-suite-data/test_parsing/n_*.json
[ERR]: failed to open json-test-suite-data/test_parsing/i_*.json

It may be a solution to clone the two repos during initial build in e.g. bootstrap.sh - or, if it's desired to only download those files when tests are actually executed, inside the actual tests.

pantoniou commented 1 year ago

https://github.com/pantoniou/libfyaml/commit/0bcced03ab3b14b69475c7e40d3e69f465e36959 should fix that.

Please test and close if it does.

schroeding commented 1 year ago

Yes, this fixes the issue! Thank you! :)