Bug: Issues with saving output using a threadpool

[adamcoulthard]

Describe the bug

I've been attempting to speed up our provisioning function, as part of the process it gets to a point where we have x number of jobs to save the output away. In the test I'm doing there are 12 jobs attempting to be saved. When the processing happens of the threads a random number of jobs are saved but not all of them. If I run them sequentially everything works as expected.

Steps to reproduce

1. Create some output on the zos machines using BatchJobs (they should be of different sizes), we have 12 pieces of output of different sizes.
2. Create a function that will use a thread pool to save away these jobs for example we use something like:

ExecutorService es = Executors.newFixedThreadPool(jobs.size());

        // Iterate through the jobs that need to be saved
        for (IZosBatchJob job : jobs)
        {
            // Start each one in a new thread
            es.submit(() ->
            {
                try
                {
                    job.saveOutputToResultsArchive(path);
                }
                catch (ZosBatchException e)
                {
                    logger.error("Failed to archive output from " + job);
                }

                try
                {
                    job.purge();
                }
                catch (ZosBatchException e)
                {
                    logger.warn("Failed to purge " + job);
                }
            });
        }

3. Run the tests (I would recommending enabling --trace on the call because this seems to cause the problem even more).

Expected behavior

I would have expected all the output to be saved away correctly.