goodboy / tractor

A distributed, structured concurrent runtime for Python (and friends)
GNU Affero General Public License v3.0
261 stars 12 forks source link

Discovery and concensus: research and discussion. #184

Open goodboy opened 3 years ago

goodboy commented 3 years ago

This is a draft issue that I hope gets filled out more and more by lurkers over time.

The road map has the following:

But in order to do these things we need both a service discovery system (presumably by relying on a (set of) protocols) and, a modern concensus system which these days is usually in the form of either a centralized algo such as raft or, the thoroughly more pumped distributed equivalents.

Here are some resources to start poking at for research and tinkering:

As always, lurkers engage.

guilledk commented 3 years ago

Crypto pump is here:

BFT options:

I'm currently working in a startup developing decentralized applications on the DPoS blockchain Telos, Its a fork of the bigger EOS blockchain.

Some good things about it:

Basically every time you need for example, voting or an escrow you can write your backend in a smart contract and interact with it from python.

The service discovery could be saved in a ledger inside a smart contract

DOCS

goodboy commented 3 years ago

thanks @guilledk, got any links on performance tradeoffs by chance?

No rush either.

gc-ss commented 3 years ago

Hashicorp has some great, single binary (so deployment is not a challenge) solutions to:

custom cross-host supervisors mult-host service architecture tooling

Hashicorp Nomad

we need both a service discovery system (presumably by relying on a (set of) protocols) and

Hashicorp Consul

a modern concensus system which these days is usually in the form of either a centralized algo such as raft

Hashicorp Consul also exposes their battle tested raft library https://github.com/hashicorp/raft

https://www.hashicorp.com/resources/everybody-talks-gossip-serf-memberlist-raft-swim-hashicorp-consul