Add multprocess queue

[mkulik-rh]

New CLI options:

• --workers - specify how many build processes spawn for batch build. Defaults to one for not multiprocess mode.
• --no-stdout - this is required for multiprocess mode. It self explanatory. Standard output printed to stdout would be very messy and not really readable due to weird order. Logger output is still printed to file.

Here is minimal stdout information in multiprocess mode:

Pool with workers is spawn at the beginning of each batch. Some data are not properly utilized in this mode like curr_comp_state because of multi process communication. Those are not needed though.

Artifacts (list or built rpms) are transferred to main process via Proxylist which is slow because serialization/de-serialization but this should not cause delays in our scenario (should be avoided still though). Standard mode works in old way.

Queue is spawn per batch.

When component fails another component in batch in queue will continue building. Error will be thrown at the end of batch build.

TODO after check:

• tests

[mkulik-rh]

I cannot measure performance because perl-bootstrap cannot be build. File in tests directory contain invalid spec file for perl package. In newer stream 5.34 some "next" packages contain invalid source file and build fails.

[mkulik-rh]

Building only 45 first components for perl:5.32 using SRPM with one worker:

5014.24s user 638.56s system 211% cpu 44:28.30 total

with 3 workers:

3990.59s user 635.27s system 285% cpu 26:59.95 total

It's quite the improvement. It would be much better because first batch contains only one component perl that's taking long time to build.

[mcurlej]

This looks good. If you did not add them, please try to write a test to run the build with multiple workers.

[mkulik-rh]

Added multiprocess marker for previous tests. Added test to cover new exceptions. Multi thread queue workers are mocked because pytest just hangs waiting for spawn processes. I'm not sure why..

[mcurlej]

+1