Implement Concurrent Workers

[mtesseract]

Let's discuss this new updated PR.

[mtesseract]

                                          +----------------------------------------+
                                          |                                        |
                                          |   Subscription Consumption             |
                                          |                                        |
                                          |     Subscription/Events.hs             |
                                          |                                        |
                                          |                                        |
                                          |  Function: subscriptionProcessHandler  |
                                          |                                        |
                                          +----------------------------------------+
                                                             |
                                                             |
                                                             |
                                       +---------------------v--------------------------+
                                       |                                                |
                                       |       Worker Access Layer                      |
                                       |                                                |
                                       |       Internal/Worker.hs                       |
                                       |                                                |
                                       |    Spawns workers, creating a Worker Registry. |
                                       |    Dispatches batches to workers.              |
                                       |    Waits for workers to terminate.             |
                                       |                                                |
                                       |                                                |
                                       +------------------------------------------------+
                                                              |
                          +------------------------------------------------------------------------+
                          |                                                                        |
                          |                                                                        |
                          |                                                                        |
                          |                                                                        |
        +-----------------v-----------------+                                    +-----------------v-----------------+
        |                                   |                                    |                                   |
        |   Worker 1                        |                                    |   Worker n                        |
        |                                   |                                    |                                   |
        |     Internal/Worker.hs            |                                    |     Internal/Worker.hs            |
        |                                   |                                    |                                   |
        |     Function: subscriptionWorker  |                                    |     Function: subscriptionWorker  |
        |                                   |                                    |                                   |
        +-----------------------------------+                                    +-----------------------------------+
                         |                                   . . . . . .                           |
                         |                                                                         |
    +--------------------v-----------------------+                          +----------------------v---------------------+
    |                                            |                          |                                            |
    |  Committer                                 |                          |  Committer                                 |
    |                                            |                          |                                            |
    |  Internal/Committer.hs                     |                          |  Internal/Committer.hs                     |
    |                                            |                          |                                            |
    |  Functions:                                |                          |  Functions:                                |
    |                                            |                          |                                            |
    |  subscriptionSink (synchronous case)       |                          |  subscriptionSink (synchronous case)       |
    |  subscriptionCommitter (asynchronous case) |                          |  subscriptionCommitter (asynchronous case) |
    |                                            |                          |                                            |
    +--------------------------------------------+                          +--------------------------------------------+

[reactormonk]

Maybe a bit late, but possibly relevant: https://github.com/roman/Haskell-capataz

[mtesseract]

@reactormonk

Thanks for your remark. Currently the code uses the async API, which — at least for me — seems to be sufficient for the use case at hand. Thus, I would see the topic of moving to another concurrency API outside of the scope of this PR. Feel free to open an issue about this.

Thanks, Moritz

[mtesseract]

Have added some more code comments in the PR.

[mtesseract]

@erthalion I have merged in master again.

[erthalion]

Other than my last question, I think I reached my limits in reviewing of this PR, so :+1:

[mtesseract]

Cool, thank you very much! I appreciate it a lot. If you consider my last answer satisfying I will go ahead with merging this stuff to master this weekend.

[erthalion]

@mtesseract yep, sure!