Peering Issues - testing branch

araskachoi commented 5 years ago

1/3/2019 Analysis:

go.sum:

mismatched checksum for the testing branch
- h1:pyowaBSivNxBr137ZjYkr0q4o41MKSJVPKuO7F7AAfY= -> h1:v/1HVWH2gq7IZGnFXrTsjoG2I5XIsU5gXiGH5SH915k=
- this was recently fixed with the master branch but now needs to be fixed on the testing branch as well

commands to run sharding application

docker exec -d whiteblock-node0 sharding-p2p-poc -seed=0 -port=5566 -rpcport=7788 ip=0.0.0.0
docker exec -d whiteblock-node1 sharding-p2p-poc -seed=1 -port=5566 -rpcport=7788 -bootstrap -bootnodes=/ip4/10.2.0.2/tcp/5566/ipfs/Qmd8vXqRiFVkcXcB1nigjCDuUyHcFkja6fQcU9w2zMzwwi -ip=0.0.0.0
docker exec -d whiteblock-node2 sharding-p2p-poc -seed=2 -port=5566 -rpcport=7788 -bootstrap -bootnodes=/ip4/10.2.0.2/tcp/5566/ipfs/Qmd8vXqRiFVkcXcB1nigjCDuUyHcFkja6fQcU9w2zMzwwi -ip=0.0.0.0
docker exec -d whiteblock-node3 sharding-p2p-poc -seed=3 -port=5566 -rpcport=7788 -bootstrap -bootnodes=/ip4/10.2.0.2/tcp/5566/ipfs/Qmd8vXqRiFVkcXcB1nigjCDuUyHcFkja6fQcU9w2zMzwwi -ip=0.0.0.0
docker exec -d whiteblock-node4 sharding-p2p-poc -seed=4 -port=5566 -rpcport=7788 -bootstrap -bootnodes=/ip4/10.2.0.2/tcp/5566/ipfs/Qmd8vXqRiFVkcXcB1nigjCDuUyHcFkja6fQcU9w2zMzwwi -ip=0.0.0.0
docker exec -d whiteblock-node5 sharding-p2p-poc -seed=5 -port=5566 -rpcport=7788 -bootstrap -bootnodes=/ip4/10.2.0.2/tcp/5566/ipfs/Qmd8vXqRiFVkcXcB1nigjCDuUyHcFkja6fQcU9w2zMzwwi -ip=0.0.0.0

bootstrap peering issue

using dockerfile from the testing branch
all docker containers can ping one another so not a networking issue
there are no peers in any of the nodes

when the client nodes use the bootstrap flag and the bootnode address when starting the application, the following error is thrown:

/bin # sharding-p2p-poc -seed=1 -port=5566 -rpcport=7788 -bootstrap -bootnodes=/ip4/10.2.0.2/tcp/5566/ipfs/Qmd8vXqRiFVkcXcB1nigjCDuUyHcFkja6fQcU9w2zMzwwi -ip=0.0.0.0
20:04:01.543 ERROR sharding-p: Failed to bootstrap with <peer.ID Qm*zMzwwi>: dial attempt failed: <peer.ID Qm*57Yrmd> --> <peer.ID Qm*zMzwwi> dial attempt failed: connection refused bootstrap.go:50

manually call `addPeer`

the bootnode can manually add peers using the cli, but when the other client nodes try to add the bootnode as a peer, the error is shown below
when the bootstrap node manually adds the peer to a client node, that client node can call addPeer to the bootstrap node without any error (but they are already peered so this is not useful to peer again)

addpeer call from QmXtW5fXrrvmHWPhq3FLHdm4zKnC5FZdhTRynSQT57Yrmd to Qmd8vXqRiFVkcXcB1nigjCDuUyHcFkja6fQcU9w2zMzwwi (node1 -> bootnode) gives error

/bin # sharding-p2p-poc -client -rpcport=7788 addpeer 10.2.0.2 5566 0
19:46:43.019 CRITI sharding-p: Failed to add peer at 10.2.0.2:5566, err: rpc error: code = Unknown desc = Failed to connect to peer <peer.ID Qm*zMzwwi>, err: dial attempt failed: <peer.ID Qm*57Yrmd> --> <peer.ID Qm*zMzwwi> dial attempt failed: connection refused cli.go:199

araskachoi commented 5 years ago

@mhchia Was there any changes made to how peering is set up with the bootnode? Or is there some type of blacklisting involved? Going to go through the source code to figure out what the issue is but would like to get any updates on your guys' end and collaborate on debugging! (:

mhchia commented 5 years ago

@araskachoi

Was there any changes made to how peering is set up with the bootnode? Or is there some type of blacklisting involved?

AFAIK we didn't, at least within these 3 weeks. cc @NIC619

Going to go through the source code to figure out what the issue is but would like to get any updates on your guys' end and collaborate on debugging! (:

Sorry haven't had a time going through the issue. Will do it, but it will definitely be great to have your help. :)

araskachoi commented 5 years ago

Found out the issue... It was because I was missing the - in the -ip=... flag! This resolves the isse and peering is working as expected!

araskachoi commented 5 years ago

for 1 boot node, 5 client nodes

 *********** Peer List ***********
node 0 peer list: ["QmaPF7a84gB8Ld35T9mJG2kDPNgQ1v5DkZr3hvjeGaxuQN","QmXtW5fXrrvmHWPhq3FLHdm4zKnC5FZdhTRynSQT57Yrmd","QmRd2F8C6Nk49zjLmNCHKRzPaMUggEicJMw6tHX6DEe8VL","QmTkBrU7gzzAZo6B93oPc2hetvv8EhmVEZGdkpmdkg7Pw2","QmYfkGxaRn675YYywLu7CHR9iTjUa7A4tHbskDcpFtPc5R"]
node 1 peer list: ["Qmd8vXqRiFVkcXcB1nigjCDuUyHcFkja6fQcU9w2zMzwwi","QmRd2F8C6Nk49zjLmNCHKRzPaMUggEicJMw6tHX6DEe8VL","QmTkBrU7gzzAZo6B93oPc2hetvv8EhmVEZGdkpmdkg7Pw2","QmYfkGxaRn675YYywLu7CHR9iTjUa7A4tHbskDcpFtPc5R","QmaPF7a84gB8Ld35T9mJG2kDPNgQ1v5DkZr3hvjeGaxuQN"]
node 2 peer list: ["QmaPF7a84gB8Ld35T9mJG2kDPNgQ1v5DkZr3hvjeGaxuQN","Qmd8vXqRiFVkcXcB1nigjCDuUyHcFkja6fQcU9w2zMzwwi","QmXtW5fXrrvmHWPhq3FLHdm4zKnC5FZdhTRynSQT57Yrmd","QmTkBrU7gzzAZo6B93oPc2hetvv8EhmVEZGdkpmdkg7Pw2","QmYfkGxaRn675YYywLu7CHR9iTjUa7A4tHbskDcpFtPc5R"]
node 3 peer list: ["Qmd8vXqRiFVkcXcB1nigjCDuUyHcFkja6fQcU9w2zMzwwi","QmRd2F8C6Nk49zjLmNCHKRzPaMUggEicJMw6tHX6DEe8VL","QmXtW5fXrrvmHWPhq3FLHdm4zKnC5FZdhTRynSQT57Yrmd","QmYfkGxaRn675YYywLu7CHR9iTjUa7A4tHbskDcpFtPc5R","QmaPF7a84gB8Ld35T9mJG2kDPNgQ1v5DkZr3hvjeGaxuQN"]
node 4 peer list: ["Qmd8vXqRiFVkcXcB1nigjCDuUyHcFkja6fQcU9w2zMzwwi","QmTkBrU7gzzAZo6B93oPc2hetvv8EhmVEZGdkpmdkg7Pw2","QmRd2F8C6Nk49zjLmNCHKRzPaMUggEicJMw6tHX6DEe8VL","QmXtW5fXrrvmHWPhq3FLHdm4zKnC5FZdhTRynSQT57Yrmd","QmaPF7a84gB8Ld35T9mJG2kDPNgQ1v5DkZr3hvjeGaxuQN"]
node 5 peer list: ["QmYfkGxaRn675YYywLu7CHR9iTjUa7A4tHbskDcpFtPc5R","Qmd8vXqRiFVkcXcB1nigjCDuUyHcFkja6fQcU9w2zMzwwi","QmRd2F8C6Nk49zjLmNCHKRzPaMUggEicJMw6tHX6DEe8VL","QmTkBrU7gzzAZo6B93oPc2hetvv8EhmVEZGdkpmdkg7Pw2","QmXtW5fXrrvmHWPhq3FLHdm4zKnC5FZdhTRynSQT57Yrmd"]

 *********** Topic Peer List ***********
node 0 topic list: {"listeningShard":["QmaPF7a84gB8Ld35T9mJG2kDPNgQ1v5DkZr3hvjeGaxuQN","QmXtW5fXrrvmHWPhq3FLHdm4zKnC5FZdhTRynSQT57Yrmd","QmRd2F8C6Nk49zjLmNCHKRzPaMUggEicJMw6tHX6DEe8VL","QmTkBrU7gzzAZo6B93oPc2hetvv8EhmVEZGdkpmdkg7Pw2","QmYfkGxaRn675YYywLu7CHR9iTjUa7A4tHbskDcpFtPc5R"]}
node 1 topic list: {"listeningShard":["Qmd8vXqRiFVkcXcB1nigjCDuUyHcFkja6fQcU9w2zMzwwi","QmRd2F8C6Nk49zjLmNCHKRzPaMUggEicJMw6tHX6DEe8VL","QmTkBrU7gzzAZo6B93oPc2hetvv8EhmVEZGdkpmdkg7Pw2","QmYfkGxaRn675YYywLu7CHR9iTjUa7A4tHbskDcpFtPc5R","QmaPF7a84gB8Ld35T9mJG2kDPNgQ1v5DkZr3hvjeGaxuQN"]}
node 2 topic list: {"listeningShard":["QmaPF7a84gB8Ld35T9mJG2kDPNgQ1v5DkZr3hvjeGaxuQN","Qmd8vXqRiFVkcXcB1nigjCDuUyHcFkja6fQcU9w2zMzwwi","QmXtW5fXrrvmHWPhq3FLHdm4zKnC5FZdhTRynSQT57Yrmd","QmTkBrU7gzzAZo6B93oPc2hetvv8EhmVEZGdkpmdkg7Pw2","QmYfkGxaRn675YYywLu7CHR9iTjUa7A4tHbskDcpFtPc5R"]}
node 3 topic list: {"listeningShard":["Qmd8vXqRiFVkcXcB1nigjCDuUyHcFkja6fQcU9w2zMzwwi","QmRd2F8C6Nk49zjLmNCHKRzPaMUggEicJMw6tHX6DEe8VL","QmXtW5fXrrvmHWPhq3FLHdm4zKnC5FZdhTRynSQT57Yrmd","QmYfkGxaRn675YYywLu7CHR9iTjUa7A4tHbskDcpFtPc5R","QmaPF7a84gB8Ld35T9mJG2kDPNgQ1v5DkZr3hvjeGaxuQN"]}
node 4 topic list: {"listeningShard":["QmTkBrU7gzzAZo6B93oPc2hetvv8EhmVEZGdkpmdkg7Pw2","QmRd2F8C6Nk49zjLmNCHKRzPaMUggEicJMw6tHX6DEe8VL","QmXtW5fXrrvmHWPhq3FLHdm4zKnC5FZdhTRynSQT57Yrmd","QmaPF7a84gB8Ld35T9mJG2kDPNgQ1v5DkZr3hvjeGaxuQN","Qmd8vXqRiFVkcXcB1nigjCDuUyHcFkja6fQcU9w2zMzwwi"]}
node 5 topic list: {"listeningShard":["Qmd8vXqRiFVkcXcB1nigjCDuUyHcFkja6fQcU9w2zMzwwi","QmRd2F8C6Nk49zjLmNCHKRzPaMUggEicJMw6tHX6DEe8VL","QmTkBrU7gzzAZo6B93oPc2hetvv8EhmVEZGdkpmdkg7Pw2","QmXtW5fXrrvmHWPhq3FLHdm4zKnC5FZdhTRynSQT57Yrmd","QmYfkGxaRn675YYywLu7CHR9iTjUa7A4tHbskDcpFtPc5R"]}

araskachoi commented 5 years ago

for 1 bootnode, 10 client nodes

 *********** Peer List ***********
node 0 peer list: ["QmXEM4JajZe6o2ejwkWA6pH3qRw3rkNbF23Lo5PJxrof7U","QmWxmXbDqhho2CrJhiu2hHFHba2bsdKDbCSV548KxxAmbB","QmeMdxpbB5aK6gYu78TdMHMLnQuEatPAw5UuRPTfoXLDsm","QmXtW5fXrrvmHWPhq3FLHdm4zKnC5FZdhTRynSQT57Yrmd","QmRd2F8C6Nk49zjLmNCHKRzPaMUggEicJMw6tHX6DEe8VL","QmTkBrU7gzzAZo6B93oPc2hetvv8EhmVEZGdkpmdkg7Pw2","QmYfkGxaRn675YYywLu7CHR9iTjUa7A4tHbskDcpFtPc5R","QmaPF7a84gB8Ld35T9mJG2kDPNgQ1v5DkZr3hvjeGaxuQN","QmRfvHwkUAnv5shq1e8mEdMjbHkuJruQgab6ijSYR5kJXv","QmUEYt4pTZwnmeiWJhCQJ2sHH46A3bs3KBVN8eN2nC87Y4"]
node 1 peer list: ["Qmd8vXqRiFVkcXcB1nigjCDuUyHcFkja6fQcU9w2zMzwwi","QmRd2F8C6Nk49zjLmNCHKRzPaMUggEicJMw6tHX6DEe8VL","QmTkBrU7gzzAZo6B93oPc2hetvv8EhmVEZGdkpmdkg7Pw2","QmaPF7a84gB8Ld35T9mJG2kDPNgQ1v5DkZr3hvjeGaxuQN","QmXEM4JajZe6o2ejwkWA6pH3qRw3rkNbF23Lo5PJxrof7U","QmUEYt4pTZwnmeiWJhCQJ2sHH46A3bs3KBVN8eN2nC87Y4","QmYfkGxaRn675YYywLu7CHR9iTjUa7A4tHbskDcpFtPc5R","QmWxmXbDqhho2CrJhiu2hHFHba2bsdKDbCSV548KxxAmbB","QmRfvHwkUAnv5shq1e8mEdMjbHkuJruQgab6ijSYR5kJXv","QmeMdxpbB5aK6gYu78TdMHMLnQuEatPAw5UuRPTfoXLDsm"]
node 2 peer list: ["Qmd8vXqRiFVkcXcB1nigjCDuUyHcFkja6fQcU9w2zMzwwi","QmXtW5fXrrvmHWPhq3FLHdm4zKnC5FZdhTRynSQT57Yrmd","QmYfkGxaRn675YYywLu7CHR9iTjUa7A4tHbskDcpFtPc5R","QmaPF7a84gB8Ld35T9mJG2kDPNgQ1v5DkZr3hvjeGaxuQN","QmXEM4JajZe6o2ejwkWA6pH3qRw3rkNbF23Lo5PJxrof7U","QmWxmXbDqhho2CrJhiu2hHFHba2bsdKDbCSV548KxxAmbB","QmRfvHwkUAnv5shq1e8mEdMjbHkuJruQgab6ijSYR5kJXv","QmTkBrU7gzzAZo6B93oPc2hetvv8EhmVEZGdkpmdkg7Pw2","QmUEYt4pTZwnmeiWJhCQJ2sHH46A3bs3KBVN8eN2nC87Y4","QmeMdxpbB5aK6gYu78TdMHMLnQuEatPAw5UuRPTfoXLDsm"]
node 3 peer list: ["QmUEYt4pTZwnmeiWJhCQJ2sHH46A3bs3KBVN8eN2nC87Y4","QmXtW5fXrrvmHWPhq3FLHdm4zKnC5FZdhTRynSQT57Yrmd","QmYfkGxaRn675YYywLu7CHR9iTjUa7A4tHbskDcpFtPc5R","QmXEM4JajZe6o2ejwkWA6pH3qRw3rkNbF23Lo5PJxrof7U","QmWxmXbDqhho2CrJhiu2hHFHba2bsdKDbCSV548KxxAmbB","QmRfvHwkUAnv5shq1e8mEdMjbHkuJruQgab6ijSYR5kJXv","Qmd8vXqRiFVkcXcB1nigjCDuUyHcFkja6fQcU9w2zMzwwi","QmRd2F8C6Nk49zjLmNCHKRzPaMUggEicJMw6tHX6DEe8VL","QmaPF7a84gB8Ld35T9mJG2kDPNgQ1v5DkZr3hvjeGaxuQN","QmeMdxpbB5aK6gYu78TdMHMLnQuEatPAw5UuRPTfoXLDsm"]
node 4 peer list: ["Qmd8vXqRiFVkcXcB1nigjCDuUyHcFkja6fQcU9w2zMzwwi","QmTkBrU7gzzAZo6B93oPc2hetvv8EhmVEZGdkpmdkg7Pw2","QmXtW5fXrrvmHWPhq3FLHdm4zKnC5FZdhTRynSQT57Yrmd","QmRd2F8C6Nk49zjLmNCHKRzPaMUggEicJMw6tHX6DEe8VL","QmWxmXbDqhho2CrJhiu2hHFHba2bsdKDbCSV548KxxAmbB","QmRfvHwkUAnv5shq1e8mEdMjbHkuJruQgab6ijSYR5kJXv","QmeMdxpbB5aK6gYu78TdMHMLnQuEatPAw5UuRPTfoXLDsm","QmaPF7a84gB8Ld35T9mJG2kDPNgQ1v5DkZr3hvjeGaxuQN","QmXEM4JajZe6o2ejwkWA6pH3qRw3rkNbF23Lo5PJxrof7U","QmUEYt4pTZwnmeiWJhCQJ2sHH46A3bs3KBVN8eN2nC87Y4"]
node 5 peer list: ["QmUEYt4pTZwnmeiWJhCQJ2sHH46A3bs3KBVN8eN2nC87Y4","Qmd8vXqRiFVkcXcB1nigjCDuUyHcFkja6fQcU9w2zMzwwi","QmXtW5fXrrvmHWPhq3FLHdm4zKnC5FZdhTRynSQT57Yrmd","QmYfkGxaRn675YYywLu7CHR9iTjUa7A4tHbskDcpFtPc5R","QmXEM4JajZe6o2ejwkWA6pH3qRw3rkNbF23Lo5PJxrof7U","QmRfvHwkUAnv5shq1e8mEdMjbHkuJruQgab6ijSYR5kJXv","QmTkBrU7gzzAZo6B93oPc2hetvv8EhmVEZGdkpmdkg7Pw2","QmRd2F8C6Nk49zjLmNCHKRzPaMUggEicJMw6tHX6DEe8VL","QmWxmXbDqhho2CrJhiu2hHFHba2bsdKDbCSV548KxxAmbB","QmeMdxpbB5aK6gYu78TdMHMLnQuEatPAw5UuRPTfoXLDsm"]
node 6 peer list: ["QmaPF7a84gB8Ld35T9mJG2kDPNgQ1v5DkZr3hvjeGaxuQN","QmRd2F8C6Nk49zjLmNCHKRzPaMUggEicJMw6tHX6DEe8VL","QmTkBrU7gzzAZo6B93oPc2hetvv8EhmVEZGdkpmdkg7Pw2","QmXtW5fXrrvmHWPhq3FLHdm4zKnC5FZdhTRynSQT57Yrmd","QmYfkGxaRn675YYywLu7CHR9iTjUa7A4tHbskDcpFtPc5R","QmWxmXbDqhho2CrJhiu2hHFHba2bsdKDbCSV548KxxAmbB","Qmd8vXqRiFVkcXcB1nigjCDuUyHcFkja6fQcU9w2zMzwwi","QmUEYt4pTZwnmeiWJhCQJ2sHH46A3bs3KBVN8eN2nC87Y4","QmeMdxpbB5aK6gYu78TdMHMLnQuEatPAw5UuRPTfoXLDsm","QmRfvHwkUAnv5shq1e8mEdMjbHkuJruQgab6ijSYR5kJXv"]
node 7 peer list: ["QmXtW5fXrrvmHWPhq3FLHdm4zKnC5FZdhTRynSQT57Yrmd","QmRfvHwkUAnv5shq1e8mEdMjbHkuJruQgab6ijSYR5kJXv","QmeMdxpbB5aK6gYu78TdMHMLnQuEatPAw5UuRPTfoXLDsm","QmRd2F8C6Nk49zjLmNCHKRzPaMUggEicJMw6tHX6DEe8VL","QmaPF7a84gB8Ld35T9mJG2kDPNgQ1v5DkZr3hvjeGaxuQN","QmTkBrU7gzzAZo6B93oPc2hetvv8EhmVEZGdkpmdkg7Pw2","QmXEM4JajZe6o2ejwkWA6pH3qRw3rkNbF23Lo5PJxrof7U","QmYfkGxaRn675YYywLu7CHR9iTjUa7A4tHbskDcpFtPc5R","QmUEYt4pTZwnmeiWJhCQJ2sHH46A3bs3KBVN8eN2nC87Y4","Qmd8vXqRiFVkcXcB1nigjCDuUyHcFkja6fQcU9w2zMzwwi"]
node 8 peer list: ["QmXtW5fXrrvmHWPhq3FLHdm4zKnC5FZdhTRynSQT57Yrmd","QmYfkGxaRn675YYywLu7CHR9iTjUa7A4tHbskDcpFtPc5R","QmWxmXbDqhho2CrJhiu2hHFHba2bsdKDbCSV548KxxAmbB","QmUEYt4pTZwnmeiWJhCQJ2sHH46A3bs3KBVN8eN2nC87Y4","Qmd8vXqRiFVkcXcB1nigjCDuUyHcFkja6fQcU9w2zMzwwi","QmXEM4JajZe6o2ejwkWA6pH3qRw3rkNbF23Lo5PJxrof7U","QmTkBrU7gzzAZo6B93oPc2hetvv8EhmVEZGdkpmdkg7Pw2","QmRd2F8C6Nk49zjLmNCHKRzPaMUggEicJMw6tHX6DEe8VL","QmaPF7a84gB8Ld35T9mJG2kDPNgQ1v5DkZr3hvjeGaxuQN","QmeMdxpbB5aK6gYu78TdMHMLnQuEatPAw5UuRPTfoXLDsm"]
node 9 peer list: ["Qmd8vXqRiFVkcXcB1nigjCDuUyHcFkja6fQcU9w2zMzwwi","QmWxmXbDqhho2CrJhiu2hHFHba2bsdKDbCSV548KxxAmbB","QmRd2F8C6Nk49zjLmNCHKRzPaMUggEicJMw6tHX6DEe8VL","QmaPF7a84gB8Ld35T9mJG2kDPNgQ1v5DkZr3hvjeGaxuQN","QmTkBrU7gzzAZo6B93oPc2hetvv8EhmVEZGdkpmdkg7Pw2","QmeMdxpbB5aK6gYu78TdMHMLnQuEatPAw5UuRPTfoXLDsm","QmYfkGxaRn675YYywLu7CHR9iTjUa7A4tHbskDcpFtPc5R","QmXEM4JajZe6o2ejwkWA6pH3qRw3rkNbF23Lo5PJxrof7U","QmRfvHwkUAnv5shq1e8mEdMjbHkuJruQgab6ijSYR5kJXv","QmXtW5fXrrvmHWPhq3FLHdm4zKnC5FZdhTRynSQT57Yrmd"]
node 10 peer list: ["QmUEYt4pTZwnmeiWJhCQJ2sHH46A3bs3KBVN8eN2nC87Y4","QmXEM4JajZe6o2ejwkWA6pH3qRw3rkNbF23Lo5PJxrof7U","QmaPF7a84gB8Ld35T9mJG2kDPNgQ1v5DkZr3hvjeGaxuQN","QmYfkGxaRn675YYywLu7CHR9iTjUa7A4tHbskDcpFtPc5R","QmXtW5fXrrvmHWPhq3FLHdm4zKnC5FZdhTRynSQT57Yrmd","QmRd2F8C6Nk49zjLmNCHKRzPaMUggEicJMw6tHX6DEe8VL","QmWxmXbDqhho2CrJhiu2hHFHba2bsdKDbCSV548KxxAmbB","QmTkBrU7gzzAZo6B93oPc2hetvv8EhmVEZGdkpmdkg7Pw2","QmRfvHwkUAnv5shq1e8mEdMjbHkuJruQgab6ijSYR5kJXv","Qmd8vXqRiFVkcXcB1nigjCDuUyHcFkja6fQcU9w2zMzwwi"]

 *********** Topic Peer List ***********
node 0 topic list: {"listeningShard":["QmXEM4JajZe6o2ejwkWA6pH3qRw3rkNbF23Lo5PJxrof7U","QmRfvHwkUAnv5shq1e8mEdMjbHkuJruQgab6ijSYR5kJXv","QmUEYt4pTZwnmeiWJhCQJ2sHH46A3bs3KBVN8eN2nC87Y4","QmeMdxpbB5aK6gYu78TdMHMLnQuEatPAw5UuRPTfoXLDsm","QmXtW5fXrrvmHWPhq3FLHdm4zKnC5FZdhTRynSQT57Yrmd","QmRd2F8C6Nk49zjLmNCHKRzPaMUggEicJMw6tHX6DEe8VL","QmTkBrU7gzzAZo6B93oPc2hetvv8EhmVEZGdkpmdkg7Pw2","QmYfkGxaRn675YYywLu7CHR9iTjUa7A4tHbskDcpFtPc5R","QmaPF7a84gB8Ld35T9mJG2kDPNgQ1v5DkZr3hvjeGaxuQN","QmWxmXbDqhho2CrJhiu2hHFHba2bsdKDbCSV548KxxAmbB"]}
node 1 topic list: {"listeningShard":["QmWxmXbDqhho2CrJhiu2hHFHba2bsdKDbCSV548KxxAmbB","QmRfvHwkUAnv5shq1e8mEdMjbHkuJruQgab6ijSYR5kJXv","QmUEYt4pTZwnmeiWJhCQJ2sHH46A3bs3KBVN8eN2nC87Y4","Qmd8vXqRiFVkcXcB1nigjCDuUyHcFkja6fQcU9w2zMzwwi","QmRd2F8C6Nk49zjLmNCHKRzPaMUggEicJMw6tHX6DEe8VL","QmXEM4JajZe6o2ejwkWA6pH3qRw3rkNbF23Lo5PJxrof7U","QmeMdxpbB5aK6gYu78TdMHMLnQuEatPAw5UuRPTfoXLDsm","QmTkBrU7gzzAZo6B93oPc2hetvv8EhmVEZGdkpmdkg7Pw2","QmYfkGxaRn675YYywLu7CHR9iTjUa7A4tHbskDcpFtPc5R","QmaPF7a84gB8Ld35T9mJG2kDPNgQ1v5DkZr3hvjeGaxuQN"]}
node 2 topic list: {"listeningShard":["QmaPF7a84gB8Ld35T9mJG2kDPNgQ1v5DkZr3hvjeGaxuQN","QmWxmXbDqhho2CrJhiu2hHFHba2bsdKDbCSV548KxxAmbB","QmRfvHwkUAnv5shq1e8mEdMjbHkuJruQgab6ijSYR5kJXv","Qmd8vXqRiFVkcXcB1nigjCDuUyHcFkja6fQcU9w2zMzwwi","QmYfkGxaRn675YYywLu7CHR9iTjUa7A4tHbskDcpFtPc5R","QmXEM4JajZe6o2ejwkWA6pH3qRw3rkNbF23Lo5PJxrof7U","QmUEYt4pTZwnmeiWJhCQJ2sHH46A3bs3KBVN8eN2nC87Y4","QmeMdxpbB5aK6gYu78TdMHMLnQuEatPAw5UuRPTfoXLDsm","QmXtW5fXrrvmHWPhq3FLHdm4zKnC5FZdhTRynSQT57Yrmd","QmTkBrU7gzzAZo6B93oPc2hetvv8EhmVEZGdkpmdkg7Pw2"]}
node 3 topic list: {"listeningShard":["QmXtW5fXrrvmHWPhq3FLHdm4zKnC5FZdhTRynSQT57Yrmd","QmXEM4JajZe6o2ejwkWA6pH3qRw3rkNbF23Lo5PJxrof7U","QmRfvHwkUAnv5shq1e8mEdMjbHkuJruQgab6ijSYR5kJXv","QmUEYt4pTZwnmeiWJhCQJ2sHH46A3bs3KBVN8eN2nC87Y4","QmeMdxpbB5aK6gYu78TdMHMLnQuEatPAw5UuRPTfoXLDsm","Qmd8vXqRiFVkcXcB1nigjCDuUyHcFkja6fQcU9w2zMzwwi","QmRd2F8C6Nk49zjLmNCHKRzPaMUggEicJMw6tHX6DEe8VL","QmYfkGxaRn675YYywLu7CHR9iTjUa7A4tHbskDcpFtPc5R","QmaPF7a84gB8Ld35T9mJG2kDPNgQ1v5DkZr3hvjeGaxuQN","QmWxmXbDqhho2CrJhiu2hHFHba2bsdKDbCSV548KxxAmbB"]}
node 4 topic list: {"listeningShard":["QmaPF7a84gB8Ld35T9mJG2kDPNgQ1v5DkZr3hvjeGaxuQN","QmWxmXbDqhho2CrJhiu2hHFHba2bsdKDbCSV548KxxAmbB","QmRfvHwkUAnv5shq1e8mEdMjbHkuJruQgab6ijSYR5kJXv","QmTkBrU7gzzAZo6B93oPc2hetvv8EhmVEZGdkpmdkg7Pw2","QmRd2F8C6Nk49zjLmNCHKRzPaMUggEicJMw6tHX6DEe8VL","QmXtW5fXrrvmHWPhq3FLHdm4zKnC5FZdhTRynSQT57Yrmd","QmeMdxpbB5aK6gYu78TdMHMLnQuEatPAw5UuRPTfoXLDsm","Qmd8vXqRiFVkcXcB1nigjCDuUyHcFkja6fQcU9w2zMzwwi","QmXEM4JajZe6o2ejwkWA6pH3qRw3rkNbF23Lo5PJxrof7U","QmUEYt4pTZwnmeiWJhCQJ2sHH46A3bs3KBVN8eN2nC87Y4"]}
node 5 topic list: {"listeningShard":["QmXtW5fXrrvmHWPhq3FLHdm4zKnC5FZdhTRynSQT57Yrmd","QmTkBrU7gzzAZo6B93oPc2hetvv8EhmVEZGdkpmdkg7Pw2","QmYfkGxaRn675YYywLu7CHR9iTjUa7A4tHbskDcpFtPc5R","QmRd2F8C6Nk49zjLmNCHKRzPaMUggEicJMw6tHX6DEe8VL","QmWxmXbDqhho2CrJhiu2hHFHba2bsdKDbCSV548KxxAmbB","Qmd8vXqRiFVkcXcB1nigjCDuUyHcFkja6fQcU9w2zMzwwi","QmXEM4JajZe6o2ejwkWA6pH3qRw3rkNbF23Lo5PJxrof7U","QmRfvHwkUAnv5shq1e8mEdMjbHkuJruQgab6ijSYR5kJXv","QmUEYt4pTZwnmeiWJhCQJ2sHH46A3bs3KBVN8eN2nC87Y4","QmeMdxpbB5aK6gYu78TdMHMLnQuEatPAw5UuRPTfoXLDsm"]}
node 6 topic list: {"listeningShard":["Qmd8vXqRiFVkcXcB1nigjCDuUyHcFkja6fQcU9w2zMzwwi","QmXtW5fXrrvmHWPhq3FLHdm4zKnC5FZdhTRynSQT57Yrmd","QmWxmXbDqhho2CrJhiu2hHFHba2bsdKDbCSV548KxxAmbB","QmRfvHwkUAnv5shq1e8mEdMjbHkuJruQgab6ijSYR5kJXv","QmUEYt4pTZwnmeiWJhCQJ2sHH46A3bs3KBVN8eN2nC87Y4","QmaPF7a84gB8Ld35T9mJG2kDPNgQ1v5DkZr3hvjeGaxuQN","QmRd2F8C6Nk49zjLmNCHKRzPaMUggEicJMw6tHX6DEe8VL","QmYfkGxaRn675YYywLu7CHR9iTjUa7A4tHbskDcpFtPc5R","QmTkBrU7gzzAZo6B93oPc2hetvv8EhmVEZGdkpmdkg7Pw2","QmeMdxpbB5aK6gYu78TdMHMLnQuEatPAw5UuRPTfoXLDsm"]}
node 7 topic list: {"listeningShard":["QmaPF7a84gB8Ld35T9mJG2kDPNgQ1v5DkZr3hvjeGaxuQN","QmYfkGxaRn675YYywLu7CHR9iTjUa7A4tHbskDcpFtPc5R","QmTkBrU7gzzAZo6B93oPc2hetvv8EhmVEZGdkpmdkg7Pw2","QmRfvHwkUAnv5shq1e8mEdMjbHkuJruQgab6ijSYR5kJXv","QmUEYt4pTZwnmeiWJhCQJ2sHH46A3bs3KBVN8eN2nC87Y4","Qmd8vXqRiFVkcXcB1nigjCDuUyHcFkja6fQcU9w2zMzwwi","QmXtW5fXrrvmHWPhq3FLHdm4zKnC5FZdhTRynSQT57Yrmd","QmXEM4JajZe6o2ejwkWA6pH3qRw3rkNbF23Lo5PJxrof7U","QmeMdxpbB5aK6gYu78TdMHMLnQuEatPAw5UuRPTfoXLDsm","QmRd2F8C6Nk49zjLmNCHKRzPaMUggEicJMw6tHX6DEe8VL"]}
node 8 topic list: {"listeningShard":["QmRd2F8C6Nk49zjLmNCHKRzPaMUggEicJMw6tHX6DEe8VL","QmaPF7a84gB8Ld35T9mJG2kDPNgQ1v5DkZr3hvjeGaxuQN","QmUEYt4pTZwnmeiWJhCQJ2sHH46A3bs3KBVN8eN2nC87Y4","QmeMdxpbB5aK6gYu78TdMHMLnQuEatPAw5UuRPTfoXLDsm","QmXtW5fXrrvmHWPhq3FLHdm4zKnC5FZdhTRynSQT57Yrmd","QmWxmXbDqhho2CrJhiu2hHFHba2bsdKDbCSV548KxxAmbB","QmXEM4JajZe6o2ejwkWA6pH3qRw3rkNbF23Lo5PJxrof7U","Qmd8vXqRiFVkcXcB1nigjCDuUyHcFkja6fQcU9w2zMzwwi","QmYfkGxaRn675YYywLu7CHR9iTjUa7A4tHbskDcpFtPc5R","QmTkBrU7gzzAZo6B93oPc2hetvv8EhmVEZGdkpmdkg7Pw2"]}
node 9 topic list: {"listeningShard":["QmWxmXbDqhho2CrJhiu2hHFHba2bsdKDbCSV548KxxAmbB","QmaPF7a84gB8Ld35T9mJG2kDPNgQ1v5DkZr3hvjeGaxuQN","QmRd2F8C6Nk49zjLmNCHKRzPaMUggEicJMw6tHX6DEe8VL","QmRfvHwkUAnv5shq1e8mEdMjbHkuJruQgab6ijSYR5kJXv","QmXEM4JajZe6o2ejwkWA6pH3qRw3rkNbF23Lo5PJxrof7U","QmXtW5fXrrvmHWPhq3FLHdm4zKnC5FZdhTRynSQT57Yrmd","Qmd8vXqRiFVkcXcB1nigjCDuUyHcFkja6fQcU9w2zMzwwi","QmTkBrU7gzzAZo6B93oPc2hetvv8EhmVEZGdkpmdkg7Pw2","QmYfkGxaRn675YYywLu7CHR9iTjUa7A4tHbskDcpFtPc5R","QmeMdxpbB5aK6gYu78TdMHMLnQuEatPAw5UuRPTfoXLDsm"]}
node 10 topic list: {"listeningShard":["QmTkBrU7gzzAZo6B93oPc2hetvv8EhmVEZGdkpmdkg7Pw2","QmaPF7a84gB8Ld35T9mJG2kDPNgQ1v5DkZr3hvjeGaxuQN","QmYfkGxaRn675YYywLu7CHR9iTjUa7A4tHbskDcpFtPc5R","QmUEYt4pTZwnmeiWJhCQJ2sHH46A3bs3KBVN8eN2nC87Y4","QmRfvHwkUAnv5shq1e8mEdMjbHkuJruQgab6ijSYR5kJXv","Qmd8vXqRiFVkcXcB1nigjCDuUyHcFkja6fQcU9w2zMzwwi","QmWxmXbDqhho2CrJhiu2hHFHba2bsdKDbCSV548KxxAmbB","QmXEM4JajZe6o2ejwkWA6pH3qRw3rkNbF23Lo5PJxrof7U","QmRd2F8C6Nk49zjLmNCHKRzPaMUggEicJMw6tHX6DEe8VL","QmXtW5fXrrvmHWPhq3FLHdm4zKnC5FZdhTRynSQT57Yrmd"]}

ethresearch / sharding-p2p-poc