tests: slow.gif content can be incorrect if tests are executed in parallel

[kmike]

Sometimes, when tests are executed in parallel, checking of slow.gif content may fail:

_________________________________________________________________________________________ HarRenderTest.test_response_body _________________________________________________________________________________________
[gw6] linux -- Python 3.5.2 /usr/bin/python3
splash/tests/test_har.py:196: in test_response_body
    assert base64.b64decode(b64_data) == img_gif
E   AssertionError: assert b'GIF89aGIF89...x02L\x01\x00;' == b'GIF89a\x01\x...x02L\x01\x00;'
E     At index 6 diff: 71 != 1
E     Left contains more items, first extra item: 2
E     Use -v to get the full diff

It seems GIF header is duplicated in the result.

[Gallaecio]

Shall we close this, given https://github.com/scrapinghub/splash/commit/dec1fccf75ee64df26cc6691085ffdd35e36f60b?

[kmike]

I'm not sure that's related.

[Gallaecio]

It changes the way tests are executed. The previous instructions suggested -n to run things in parallel; now https://github.com/scrapinghub/splash/blob/master/dockerfiles/tests/runtests.sh runs the tests, which does not use -n. So, following the current instructions to run the tests, you cannot get this issue.

So, I’m wondering if instead of fixing the issue, we could either ignore it and assume people won’t try to execute tests in parallel, or explicitly document that tests do not support parallel execution.

[kmike]

You can still run tests in Docker in parallel, arguments like -n are passed to the command. It is super-useful to have these tests in parallel, as they're quite long; I recall doing it with -n4 or -n8 always.