Currently we use a job on the background queue for each run separately to collect the logs from the docker containers as they run.
This was necessary as the only way to collect logs in the original version of docker we were using way back when was to attach the container.
However, now we can centralise logging by using a log server which docker supports in many variants. This means that we can centralise the collection of logs. Which should improve performance and simplify things considerably.
However, it is a pretty sizable re-engineering effort. But I think one that would be worth it
Currently we use a job on the background queue for each run separately to collect the logs from the docker containers as they run.
This was necessary as the only way to collect logs in the original version of docker we were using way back when was to attach the container.
However, now we can centralise logging by using a log server which docker supports in many variants. This means that we can centralise the collection of logs. Which should improve performance and simplify things considerably.
However, it is a pretty sizable re-engineering effort. But I think one that would be worth it