Load Balancing General implementation

Implementation of a general load balancing pattern.

We will one Client generating tasks connected to a Broker Proxy which distributes the work across workers. After each worker finish their job they send the results to a Sink. This Sink returns the results back to the Client.

The Proxy must distribute the tasks evenly between workers and in case of worker failure its remaining tasks will be sent to other workers.

Install

pipenv install git+https://github.com/mattborghi/load-balancer-zmq/#egg=LoadBalancer

or

pipenv update

if the package is already present on the Pipenv file.

Local development instructions

Set working environment

pipenv install
pipenv shell

A. Together

python main.py

B. Separately

Run controller

python LoadBalancer/controller.py

Run this line for each worker you want to spawn in a separater terminal

python LoadBalancer/worker.py

Run Sink

python LoadBalancer/sink.py

Generate release files with name based on setup.py

python setup.py sdist bdist_wheel