search

liftbridge-io / go-liftbridge

Go client for Liftbridge. https://github.com/liftbridge-io/liftbridge
Apache License 2.0
66 stars 18 forks source link

Redirect client to a better broker based on specific criterias #114

Closed LaPetiteSouris closed 2 years ago

LaPetiteSouris commented 2 years ago

Motivation

Required: https://github.com/liftbridge-io/liftbridge-api/pull/53 and https://github.com/liftbridge-io/liftbridge/pull/362

To solve: https://github.com/liftbridge-io/liftbridge/issues/274

Solution

  1. Broker Selection In broker, add the state for each broker. The state contains PartitionCount and LastKnownLatency.
    • PartitionCount represents the sum of all partitions hosted on this server . I.e: 
partitionCount = SUM( partitions on this server + partitions that have this server as their leader)
  1. Selection config The client can choose to connect to a broker based on either SetConnectToLowWorkLoadServer() or SetConnectionToLowLatencyServer() to decide a better server it shall connect to. If nothing is chosen, then a random broker will be given (default behavior)
LaPetiteSouris commented 2 years ago

The build in CI looks a bit weird to me.

#!/bin/bash -eo pipefail
sudo apt-get update -yqq && sudo apt-get install -yqq bzr
E: Repository 'http://security.debian.org/debian-security buster/updates InRelease' changed its 'Suite' value from 'stable' to 'oldstable'
E: Repository 'http://deb.debian.org/debian buster InRelease' changed its 'Suite' value from 'stable' to 'oldstable'
E: Repository 'http://deb.debian.org/debian buster-updates InRelease' changed its 'Suite' value from 'stable-updates' to 'oldstable-updates'

Exited with code exit status 100
CircleCI received exit code 100
tylertreat commented 2 years ago

Regarding the failed CI, it looks like the circleci docker image probably needs to be updated.