mosaicnetworks / babble

Distributed Consensus Middleware
MIT License
477 stars 95 forks source link

C++ example? #98

Closed Outstep closed 5 years ago

Outstep commented 5 years ago

Hello,

I have just come across Babble (Golang) which might be very useful for a project that I am working on which is composed of a high-speed graph database library that I wish to make distributed but which is written in C++ and am testing on Ubuntu 18.0.

Do you have a c++ example?

Also, if I have read correctly then is Babble basically allowing for Sharding across nodes which basically means that it replicates commands to have the same datastores on different physical machines (i.e. distributed nodes in a cryptocurrency system)?

Cheers :)

arrivets commented 5 years ago

Hi,

Sounds interesting. We don't have any c++ examples yet, sorry.

Yes Babble allows creating replicated datastores, by ensuring that all replicas process the same transactions in the same order. For example we are using it with the Ethereum Virtual Machine to create mini Ethereum networks/ shards: https://github.com/mosaicnetworks/evm-lite

Sounds like your project is similar to https://github.com/dgraph-io/dgraph. Am I right?

They are using Raft consensus for the replication aspect. Babble is a good alternative :)

Outstep commented 5 years ago

Hello,

Thanks for getting back to me so quickly.

You are correct in part of what I am doing as one project based upon these Graph Database libraries is to build them out into a very solid distributed graph database platform that has a number of features and would be competitors to DGraph plus a number of features that they do not offer.

Originally, I had been looking at the possibility of using OpenDHT (https://github.com/savoirfairelinux/opendht) to add in the distributed functionality as it seems rather easy to integrate, and also would allow for actually spreading out the data graphs across multiple nodes and not just sharding for replicates but I also think that I read that OpenDHT was a bit slow as well which may not work well for that project.

The other project that I am currently integrating the graph database libraries into as a type of hypergraph crypto blockchain "hypergraph-chain" instead of a blockchain (bitcoin, litecoin, etc....,) or lattice-chain (Raiblocks now known as NANO coin) in which I think that the hypergraph chain could be significantly faster in the Transactions-Per-Second area and your Babble code could be really useful in this project since it is about replicating nodes as well.

Cheers, :)

Outstep commented 5 years ago

Hello Martin,

I actually tried to follow your installation and demo 2 times, but in each case they failed with babble container exiting:


babble-master$ docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES b9876f3817c4 mosaicnetworks/watcher "/watch.sh 4" 13 minutes ago Up 13 minutes watcher 458497ff0e71 mosaicnetworks/babble:0.4.0 "babble run --cache-…" 13 minutes ago Exited (2) 13 minutes ago node4 43d919049774 mosaicnetworks/babble:0.4.0 "babble run --cache-…" 13 minutes ago Exited (2) 13 minutes ago node3 5b9ea627fc13 mosaicnetworks/babble:0.4.0 "babble run --cache-…" 13 minutes ago Exited (2) 13 minutes ago node2 263d9714e1d6 mosaicnetworks/babble:0.4.0 "babble run --cache-…" 13 minutes ago Exited (2) 13 minutes ago node1 6d6807ea37ff mosaicnetworks/dummy:0.4.0 "dummy '--name=clien…" 13 minutes ago Up 13 minutes 1339/tcp client4 cca902fbeaa9 mosaicnetworks/dummy:0.4.0 "dummy '--name=clien…" 13 minutes ago Up 13 minutes 1339/tcp client3 839c2604b98f mosaicnetworks/dummy:0.4.0 "dummy '--name=clien…" 14 minutes ago Up 13 minutes 1339/tcp client2 0714c7a903ca mosaicnetworks/dummy:0.4.0 "dummy '--name=clien…" 14 minutes ago Up 14 minutes 1339/tcp client1


The logs for one of the exited container shows:

docker logs 458497ff0e71 time="2019-01-18T22:51:44Z" level=debug msg=RUN ClientAddr="172.77.5.8:1339" ProxyAddr="172.77.5.4:1338" Standalone=false babble.BindAddr="172.77.5.4:1337" babble.DataDir=/.babble babble.LoadPeers=true babble.LogLevel=debug babble.MaxPool=2 babble.Node.HeartbeatTimeout=10ms babble.Node.TCPTimeout=200ms babble.ServiceAddr="172.77.5.4:80" babble.Store=true babble.node.CacheSize=50000 babble.node.SyncLimit=1000 panic: runtime error: slice bounds out of range

goroutine 1 [running]: github.com/mosaicnetworks/babble/src/peers.(Peer).PubKeyBytes(0xc42013dfb0, 0x7f8b0d9a9d90, 0x0, 0xc42019da50, 0x410ed8, 0x30) /home/martin/gowork/src/github.com/mosaicnetworks/babble/src/peers/peer.go:31 +0x98 github.com/mosaicnetworks/babble/src/peers.(Peer).computeID(0xc42013dfb0, 0x40, 0xb35160) /home/martin/gowork/src/github.com/mosaicnetworks/babble/src/peers/peer.go:36 +0x2f github.com/mosaicnetworks/babble/src/peers.(Peers).addPeerRaw(0xc420153e80, 0xc42013dfb0) /home/martin/gowork/src/github.com/mosaicnetworks/babble/src/peers/peers.go:47 +0xe5 github.com/mosaicnetworks/babble/src/peers.NewPeersFromSlice(0xc42013fa40, 0x4, 0x4, 0x0) /home/martin/gowork/src/github.com/mosaicnetworks/babble/src/peers/peers.go:31 +0x9b github.com/mosaicnetworks/babble/src/peers.(JSONPeers).Peers(0xc42013f9e0, 0x0, 0x0, 0x0) /home/martin/gowork/src/github.com/mosaicnetworks/babble/src/peers/json_peers.go:50 +0x1e4 github.com/mosaicnetworks/babble/src/babble.(Babble).initPeers(0xc42019dc80, 0x0, 0x0) /home/martin/gowork/src/github.com/mosaicnetworks/babble/src/babble/babble.go:62 +0xb8 github.com/mosaicnetworks/babble/src/babble.(Babble).Init(0xc42019dc80, 0xf, 0x7fff91a99f17) /home/martin/gowork/src/github.com/mosaicnetworks/babble/src/babble/babble.go:176 +0x42 github.com/mosaicnetworks/babble/cmd/babble/commands.runBabble(0xc4201a8f00, 0xc4201a50e0, 0x0, 0xa, 0x0, 0x0) /home/martin/gowork/src/github.com/mosaicnetworks/babble/cmd/babble/commands/run.go:51 +0xd0 github.com/mosaicnetworks/babble/vendor/github.com/spf13/cobra.(Command).execute(0xc4201a8f00, 0xc42001c0e0, 0xa, 0xa, 0xc4201a8f00, 0xc42001c0e0) /home/martin/gowork/src/github.com/mosaicnetworks/babble/vendor/github.com/spf13/cobra/command.go:762 +0x468 github.com/mosaicnetworks/babble/vendor/github.com/spf13/cobra.(Command).ExecuteC(0xfab720, 0x2, 0xc4201a8f00, 0xfab870) /home/martin/gowork/src/github.com/mosaicnetworks/babble/vendor/github.com/spf13/cobra/command.go:852 +0x30a github.com/mosaicnetworks/babble/vendor/github.com/spf13/cobra.(*Command).Execute(0xfab720, 0xc42019df60, 0x3) /home/martin/gowork/src/github.com/mosaicnetworks/babble/vendor/github.com/spf13/cobra/command.go:800 +0x2b main.main() /home/martin/gowork/src/github.com/mosaicnetworks/babble/cmd/babble/main.go:21 +0xaf

All of the exited containers have similar logs. Any ideas Thanks,

arrivets commented 5 years ago

hmm I've never seen this before. Are you running on a 32bit machine?

There is a known issue in the master branch: https://github.com/mosaicnetworks/babble/commit/62aa47307c84f56817d2522a6bef7ef49c561847

That's the first thing that comes to mind.

It's fixed on the develop branch.

Outstep commented 5 years ago

Hi,

I am running Ubuntu 18.04 (64 bit).

Will try again today to see what might be happening. Cheers,

arrivets commented 5 years ago

Any luck with this?