search

openaustralia / yinyo

A wonderfully simple API driven service to reliably execute many long running scrapers in a super scaleable way
https://yinyo.io
Apache License 2.0
6 stars 1 forks source link

Add some simulated failures to python test client #74

Closed jamezpolley closed 4 years ago

jamezpolley commented 4 years ago

This gives output on both stderr and stdout with some obvious hints about the expected ordering. This lets us add some tests to confirm that the output comes out as expected.

I haven't been able to test this at current head because I don't have the right go version installed and looking at the docs on how to get that set up is getting me confused so I'm not able to run the new clay client yet.

jamezpolley commented 4 years ago

@mlandauer have you had a chance to look at this?

mlandauer commented 4 years ago

Because the stdout and stderr streams truly happen in parallel there is no way to absolutely guarantee their relative order. So, it's tricky to make a test case that shows problems while at the same time works reliably. In the meantime we've fixed the stdout/stderr ordering problem in https://github.com/openaustralia/yinyo/commit/fc4ae55f3bc9a2dab21c6d56a6e275fbad4b9c69