`make check -j16` fails due to .out file collisions

[trofi]

Noticed when ran package tests as part of nixpkgs packaging.

Why things fail

I think the failure happens because multiple tests use the same output file name which collide on one another. For example if we pick 5.plist there are 2 test that use $top_builddir/test/data/5.plist.out file:

test/huge++.test-DATAOUT=$top_builddir/test/data
test/huge++.test:TESTFILE=5.plist
...
test/huge++.test-$top_builddir/test/plist_test++ $DATASRC/$TESTFILE $DATAOUT/$TESTFILE.out

test/huge.test-DATAOUT=$top_builddir/test/data
test/huge.test:TESTFILE=5.plist
...
test/huge.test-$top_builddir/test/plist_test $DATASRC/$TESTFILE $DATAOUT/$TESTFILE.out

How things fail

$ make check -j16 VERBOSE=y
...
============================================================================
Testsuite summary for libplist 2.3.0
============================================================================
# TOTAL: 41
# PASS:  40
# SKIP:  0
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0
============================================================================
See test/test-suite.log
Please report to https://github.com/libimobiledevice/libplist/issues
============================================================================

Looking at test/test-suite.log:

=========================================
   libplist 2.3.0: test/test-suite.log
=========================================

# TOTAL: 41
# PASS:  40
# SKIP:  0
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0

.. contents:: :depth: 2

FAIL: huge
==========

Converting
File ../test/data/5.plist is open
PList XML parsing succeeded
PList BIN writing succeeded
PList BIN parsing succeeded
PList XML writing succeeded
Size of input and output is different
Input size : 4292380
Output size : 4305301
Comparing
PList parsing failed
FAIL huge.test (exit status: 3)

[trofi]

What would be the best way to fix it? Tweak $TESTOUT slightly? Or perhaps run each test in it's own directory?

[nikias]

Well I guess we should just rename each file to be unique after all. The input file can stay the same (e.g. 5.plist) but the output file should be huge.out.plist, huge++.out.plist, etc.

[trofi]

Proposed possible fix as https://github.com/libimobiledevice/libplist/pull/235