Experiment over a long period

iamyulong commented 5 years ago

Scale to 16/32 nodes

iamyulong commented 5 years ago

Experiment setup:

2 nodes (in Toronto office)
Each node is running a GPU miner and a staker (190 sol/s and 300 sol/s)
The two nodes are collected via a local network.

iamyulong commented 5 years ago

Unfortunately, the second machine died again for unclear reason. A total of 23000 blocks have been generated during the weekend. Below are the stats based block [10000, 20000].

Block time:

Latest block number: 23015
Fetching data from block #10000 to #20000
Block time statics ->  (Mean: 10.26, Std: 12.37)
Pow Block time statics ->  (Mean: 20.11, Std: 20.63)
Pos Block time statics ->  (Mean: 20.93, Std: 23.19)

PoW difficulty:

PoS difficulty:

Orphaned block rate:

Latest block number: 23067
Fetching data from block #10000 to #20000
Number of orphaned blocks: 272
Orphaned block rate: 0.027197280271972803

Block reward distribution:

Latest block number: 23076
Fetching data from block #10000 to #20000
total pos:4903 --- 49.02509749025097%
total pow:5098 --- 50.97490250974902%
                                  Key                                Type    Total Block     %overall
0xa0bd75fcD7676504671EE75F95e1ED7ada6D168d1b852956568e1a32cE6a7886  Pos 4903    49.02509749025097%
0x1111111111111111111111111111111111111111111111111111111111111111  Pow 5098    50.97490250974902%

Total difficulty (accumulated PoW diff x accumulated PoS diff):

iamyulong commented 5 years ago

Blockade docker cluster set up:

16 nodes;
8 miners (1 CPU thread each);
8 stakers (1000, 1000, 2000, 2000, 3000, 3000, 4000, 4000);
The connectivity between nodes is SLOW as defined in blockade.

The reduce CPU load, I'm plugging in miners from an external machine.

iamyulong commented 5 years ago

Stats from block [2000, 7000].

Block time:

Latest block number: 7438
Fetching data from block #2000 to #7000
Block time statics ->  (Mean: 8.32, Std: 6.78)
Pow Block time statics ->  (Mean: 14.05, Std: 11.80)
Pos Block time statics ->  (Mean: 20.38, Std: 21.05)

Orphaned block rate:

Latest block number: 7452
Fetching data from block #2000 to #7000
Number of orphaned blocks: 593
Orphaned block rate: 0.11857628474305139

Block rewards:

[http://10.0.4.47:19001]Latest block number: 7453
[http://10.0.4.47:19001]Fetching data from block #2000 to #7000
[http://10.0.4.47:19002]Latest block number: 7453
[http://10.0.4.47:19002]Fetching data from block #2000 to #7000
[http://10.0.4.47:19001]total pos:2039 --- 40.77184563087383%
[http://10.0.4.47:19001]total pow:2962 --- 59.22815436912617%
                                  Key                                Type    Total Block     %overall
[http://10.0.4.47:19001]0x0000000800000008000000080000000800000008000000080000000800000008  Pow 378 7.558488302339532%
[http://10.0.4.47:19001]0x0000000300000003000000030000000300000003000000030000000300000003  Pow 370 7.398520295940812%
[http://10.0.4.47:19001]0x0000000200000002000000020000000200000002000000020000000200000002  Pow 386 7.7184563087382525%
[http://10.0.4.47:19001]0x0000000500000005000000050000000500000005000000050000000500000005  Pow 324 6.47870425914817%
[http://10.0.4.47:19001]0xA0b4F4238780C75B8CB85FA545bBD2B6BA41213a7dE694943bc06ebb4ae4f4D9  Pos 101 2.0195960807838436%
[http://10.0.4.47:19001]0x0000000600000006000000060000000600000006000000060000000600000006  Pow 371 7.418516296740652%
[http://10.0.4.47:19001]0x0000000700000007000000070000000700000007000000070000000700000007  Pow 390 7.798440311937613%
[http://10.0.4.47:19001]0x0000000100000001000000010000000100000001000000010000000100000001  Pow 374 7.478504299140172%
[http://10.0.4.47:19001]0xa02121d0a17fDb1fB3bd14a13B68db983417bD42e7D3B0C7916551BFD9443E37  Pos 315 6.29874025194961%
[http://10.0.4.47:19001]0x0000000400000004000000040000000400000004000000040000000400000004  Pow 369 7.378524295140972%
[http://10.0.4.47:19001]0xA045A87e03322563474d4dF61D805d7a945578aA402Ba43905a8aFDB590951D5  Pos 292 5.83883223355329%
[http://10.0.4.47:19001]0xa0b9166eefb20B8317006cC8A691781B23a836A94CC8B5E17598d567506a7160  Pos 399 7.978404319136173%
[http://10.0.4.47:19001]0xA040d1069a70DFC82E6Ac58cFBB4520F1b542FA2554EC3D6300A16B7f9d8F51D  Pos 424 8.478304339132174%
[http://10.0.4.47:19001]0xa0A6795Cf6106e1a1C886e05e956a81533ba9f3212Eeb64240E1d74FF57a4324  Pos 115 2.299540091981604%
[http://10.0.4.47:19001]0xA0cFf27cf76784df14Ab1e13CBe773BcF77d191b93653d2a0688B07958f19A37  Pos 206 4.119176164767047%
[http://10.0.4.47:19001]0xA0ca529Ee50bf27870D9655bc847c0a6d042523442C7D671ac5C4F6Ba148aBA6  Pos 187 3.739252149570086%

PoW difficulty:

PoS difficulty:

Total difficulty:

iamyulong commented 5 years ago

Block rewards:

Node	Power (cpu or stake)	Rewards (# of blocks)
1	1	374
2	1	386
3	1	370
4	1	369
5	1	324
6	1	371
7	1	390
8	1	378
9	1000	101
10	1000	115
11	2000	187
12	2000	206
13	3000	292
14	3000	315
15	4000	424
16	4000	399

iamyulong commented 5 years ago

Trying to reproduce the experiment:

Node 1: 1 GPU miner (190 Sol/s) + 1000 wei stake Node 2: 1 GPU miner (120 Sol/s) + 2000 wei stake

iamyulong commented 5 years ago

Stats from block [2000, 7000]:

Block time:

Latest block number: 7462
Fetching data from block #2000 to #7000
Block time statics ->  (Mean: 9.99, Std: 11.88)
Pow Block time statics ->  (Mean: 19.49, Std: 19.97)
Pos Block time statics ->  (Mean: 20.48, Std: 21.98)

Orphaned block rate:

Latest block number: 7471
Fetching data from block #2000 to #7000
Number of orphaned blocks: 146
Orphaned block rate: 0.029194161167766446

Rewards:

Latest block number: 7473
Fetching data from block #2000 to #7000
total pos:2438 --- 48.75024995001%
total pow:2563 --- 51.249750049990006%
                                  Key                                Type    Total Block     %overall
0x1111111111111111111111111111111111111111111111111111111111111111  Pow 1632    32.63347330533893%
0xa0bd75fcD7676504671EE75F95e1ED7ada6D168d1b852956568e1a32cE6a7886  Pos 762 15.236952609478106%
0x2222222222222222222222222222222222222222222222222222222222222222  Pow 931 18.61627674465107%
0xA08c895FC144884e989a32a7CbFBf47346aD3926F57635A10F10C24b09135CA3  Pos 1676    33.51329734053189%

PoW difficulty:

PoS difficulty: PoW diff

Total difficulty: PoW diff

iamyulong commented 5 years ago

I'm setting the stake of the two nodes to [10000, 20000] at block 7605 and 7605 respectively.

iamyulong commented 5 years ago

Similar results were observed despite the significant stake increase.

Now exploring the following questions:

If there are many miners with similar hash rates, will the orphaned block rate be high?
If there are many stakers with similar stakes, will the orphaned block rate be high?

Continue experiments with the following setup:

16 nodes;
8 miners, each with 1 CPU thread;
8 stakers, each with 1000 wei stake;
Network condition is "slow".

iamyulong commented 5 years ago

We had a few assumptions over why the PoW block time is short.

Staking is affecting mining, or vice versa
Hashrate is too low
Hashrate or stake power is too close

Ge is exploring no. 1 and Fred is exploring no. 2

ghost commented 5 years ago

Experiment One : 4 nodes, 4 miners with 1 thread each, 0 stakers

Latest block number: 1016
Fetching data from block #201 to #1000
Block time statics ->  (Mean: 18.56, Std: 16.37)
Pow Block time statics ->  (Mean: 18.56, Std: 16.37)
Pos Block time statics ->  (Mean: -1, Std: -1)

[http://127.0.0.1:9001]Latest block number: 1009
[http://127.0.0.1:9001]Fetching data from block #201 to #1000
[http://127.0.0.1:9002]Latest block number: 1009
[http://127.0.0.1:9002]Fetching data from block #201 to #1000
[http://127.0.0.1:9001]total pos:0 --- 0%
[http://127.0.0.1:9001]total pow:800 --- 100%
                                  Key                                Type    Total Block     %overall
[http://127.0.0.1:9001]0x2222222222222222222222222222222222222222222222222222222222222222   Pow 189 23.625%
[http://127.0.0.1:9001]0x1111111111111111111111111111111111111111111111111111111111111111   Pow 205 25.624999999999996%
[http://127.0.0.1:9001]0x3333333333333333333333333333333333333333333333333333333333333333   Pow 209 26.125%
[http://127.0.0.1:9001]0x4444444444444444444444444444444444444444444444444444444444444444   Pow 197 24.625%

Latest block number: 1020
Fetching data from block #201 to #1000
Number of orphaned blocks: 54
Orphaned block rate: 0.0675

pow diff

total diff

Experiment Two : 4 nodes, 0 miners, 4 stakers, 10000 wei each

Latest block number: 2163
Fetching data from block #1001 to #2000
Block time statics ->  (Mean: 20.69, Std: 22.17)
Pow Block time statics ->  (Mean: -1, Std: -1)
Pos Block time statics ->  (Mean: 20.69, Std: 22.17)

[http://127.0.0.1:9001]Latest block number: 2173
[http://127.0.0.1:9001]Fetching data from block #1001 to #2000
[http://127.0.0.1:9001]total pos:1000 --- 100%
[http://127.0.0.1:9001]total pow:0 --- 0%
                                  Key                                Type    Total Block     %overall
[http://127.0.0.1:9001]0xa0D9342Bc958587c8f14781EB6B124f68336d3921732a111343f11DF0E3F13Fb   Pos 244 24.4%
[http://127.0.0.1:9001]0xA06586f27e6C4e218183CdE720931B35056d3857B52B8aA28afbF0dB110cAC03   Pos 228 22.8%
[http://127.0.0.1:9001]0xa0bd75fcD7676504671EE75F95e1ED7ada6D168d1b852956568e1a32cE6a7886   Pos 267 26.700000000000003%
[http://127.0.0.1:9001]0xA08c895FC144884e989a32a7CbFBf47346aD3926F57635A10F10C24b09135CA3   Pos 261 26.1%

Latest block number: 2188
Fetching data from block #1001 to #2000
Number of orphaned blocks: 20
Orphaned block rate: 0.02

pow diff

total diff

iamyulong commented 5 years ago

Configuration:

16 nodes
8 miners, with 1 CPU thread each;
8 stakers, with 1000 wei each;

Block time:

Latest block number: 9915
Fetching data from block #2000 to #8000
Block time statics ->  (Mean: 7.87, Std: 6.07)
Pow Block time statics ->  (Mean: 12.68, Std: 9.70)
Pos Block time statics ->  (Mean: 20.75, Std: 22.63)

Orphaned block rate:

Latest block number: 9943
Fetching data from block #2000 to #8000
Number of orphaned blocks: 776
Orphaned block rate: 0.1293117813697717

Rewards:

[http://10.0.4.47:19001]Latest block number: 9951
[http://10.0.4.47:19001]Fetching data from block #2000 to #8000
[http://10.0.4.47:19001]total pos:2274 --- 37.89368438593568%
[http://10.0.4.47:19001]total pow:3727 --- 62.10631561406432%
                                  Key                                Type    Total Block     %overall
[http://10.0.4.47:19001]0xA0cFf27cf76784df14Ab1e13CBe773BcF77d191b93653d2a0688B07958f19A37  Pos 247 4.115980669888352%
[http://10.0.4.47:19001]0xA045A87e03322563474d4dF61D805d7a945578aA402Ba43905a8aFDB590951D5  Pos 281 4.682552907848692%
[http://10.0.4.47:19001]0x0000000500000005000000050000000500000005000000050000000500000005  Pow 461 7.682052991168138%
[http://10.0.4.47:19001]0x0000000800000008000000080000000800000008000000080000000800000008  Pow 434 7.232127978670222%
[http://10.0.4.47:19001]0x0000000400000004000000040000000400000004000000040000000400000004  Pow 474 7.898683552741209%
[http://10.0.4.47:19001]0xa02121d0a17fDb1fB3bd14a13B68db983417bD42e7D3B0C7916551BFD9443E37  Pos 316 5.2657890351608065%
[http://10.0.4.47:19001]0x0000000600000006000000060000000600000006000000060000000600000006  Pow 494 8.231961339776705%
[http://10.0.4.47:19001]0xA0b4F4238780C75B8CB85FA545bBD2B6BA41213a7dE694943bc06ebb4ae4f4D9  Pos 299 4.9825029161806365%
[http://10.0.4.47:19001]0x0000000300000003000000030000000300000003000000030000000300000003  Pow 459 7.64872521246459%
[http://10.0.4.47:19001]0x0000000200000002000000020000000200000002000000020000000200000002  Pow 487 8.115314114314282%
[http://10.0.4.47:19001]0x0000000700000007000000070000000700000007000000070000000700000007  Pow 442 7.365439093484419%
[http://10.0.4.47:19001]0x0000000100000001000000010000000100000001000000010000000100000001  Pow 476 7.932011331444759%
[http://10.0.4.47:19001]0xa0b9166eefb20B8317006cC8A691781B23a836A94CC8B5E17598d567506a7160  Pos 290 4.8325279120146645%
[http://10.0.4.47:19001]0xa0A6795Cf6106e1a1C886e05e956a81533ba9f3212Eeb64240E1d74FF57a4324  Pos 278 4.632561239793368%
[http://10.0.4.47:19001]0xA040d1069a70DFC82E6Ac58cFBB4520F1b542FA2554EC3D6300A16B7f9d8F51D  Pos 298 4.965839026828862%
[http://10.0.4.47:19001]0xA0ca529Ee50bf27870D9655bc847c0a6d042523442C7D671ac5C4F6Ba148aBA6  Pos 265 4.415930678220296%

I don't think it's because the power of nodes is too close, as the staking is not being affected.

fredericzha commented 5 years ago

Configuration: 8 nodes 4 miners with 1 EWBF miner each. GPU: GTX 1050 4 stakers with stake {1000, 2000, 3000, 4000}

Difficulty adjustment: before commit 272da4512f478136f26b509f613e5ccb41d025f4 alpha = 0.1 without minimal 1 difficulty adjustment in each step

Block time

Latest block number: 3617
Fetching data from block #200 to #3000
Block time statics ->  (Mean: 10.45, Std: 12.66)
Pow Block time statics ->  (Mean: 21.01, Std: 22.67)
Pos Block time statics ->  (Mean: 20.72, Std: 22.89)

Rewards distribution

Latest block number: 3736
Fetching data from block #200 to #3736
total pos:1760 --- 49.759683347469604%
total pow:1777 --- 50.240316652530396%
                                  Key                            Type    Total Block     %overall
0xa0bd75fcD7676504671EE75F95e1ED7ada6D168d1b852956568e1a32cE6a7886  Pos 159 4.495%
0xA08c895FC144884e989a32a7CbFBf47346aD3926F57635A10F10C24b09135CA3  Pos 391 11.055%
0xA06586f27e6C4e218183CdE720931B35056d3857B52B8aA28afbF0dB110cAC03  Pos 544 15.380%
0xa0D9342Bc958587c8f14781EB6B124f68336d3921732a111343f11DF0E3F13Fb  Pos 666 18.830%
0x5555555555555555555555555555555555555555555555555555555555555555  Pow 438 12.383%
0x6666666666666666666666666666666666666666666666666666666666666666  Pow 432 12.214%
0x7777777777777777777777777777777777777777777777777777777777777777  Pow 449 12.694%
0x8888888888888888888888888888888888888888888888888888888888888888  Pow 458 12.949%

fredericzha commented 5 years ago

Configuration: 8 nodes 4 miners with 1 EWBF miner each. GPU: GTX 1050 4 stakers with stake {1000, 2000, 3000, 4000}

Difficulty adjustment: after commit 0d235f1c60ec7f35f415a4c53a54bb97f34b5830 alpha = 0.05 with minimal 1 difficulty adjustment in each step

Block time

Latest block number: 2567
Fetching data from block #500 to #2500
Block time statics ->  (Mean: 9.98, Std: 11.70)
Pow Block time statics ->  (Mean: 19.48, Std: 19.18)
Pos Block time statics ->  (Mean: 20.46, Std: 21.26)

Rewards distribution

Latest block number: 2598
Fetching data from block #500 to #2500
total pos:976 --- 48.77561219390305%
total pow:1025 --- 51.22438780609695%
                                  Key                                Type    Total Block     %overall
0xa0bd75fcD7676504671EE75F95e1ED7ada6D168d1b852956568e1a32cE6a7886  Pos 110 5.497%
0xA08c895FC144884e989a32a7CbFBf47346aD3926F57635A10F10C24b09135CA3  Pos 185 9.245%
0xA06586f27e6C4e218183CdE720931B35056d3857B52B8aA28afbF0dB110cAC03  Pos 271 13.543%
0xa0D9342Bc958587c8f14781EB6B124f68336d3921732a111343f11DF0E3F13Fb  Pos 410 20.490%
0x5555555555555555555555555555555555555555555555555555555555555555  Pow 228 11.394%
0x6666666666666666666666666666666666666666666666666666666666666666  Pow 236 11.794%
0x7777777777777777777777777777777777777777777777777777777777777777  Pow 275 13.743%
0x8888888888888888888888888888888888888888888888888888888888888888  Pow 286 14.293%

Orphaned block rate (only getting level 1 orphans from the view of a single node)

Latest block number: 2571
Fetching data from block #500 to #2500
Number of orphaned blocks: 51
Orphaned block rate: 0.025487256371814093

Orphaned block rate (getting all orphans from the view of a single node)

Latest block number: 2580
Fetching data from block #500 to #2500
Main chain blocks count: 2001
Orpahned blocks count: 51 POW: 26 POS: 25
Orpahned blocks rate: 0.025 POW: 0.51 POS: 0.49

Difficulty adjustment: POW pow diff

POS pow diff

fredericzha commented 5 years ago

Updates after running 3 days, at block height 26000.

One EWBF miner died at around 5500 block height due to "out of memory" error. It is a Ubuntu 16.04 machine (others are 18.04), and is using an old version of EWBF miner for Cuda 8 because of driver version support limit.

Block time from blocks 500 to 5500 (with 4 miners):

Latest block number: 26387
Fetching data from block #500 to #5500
Block time statics ->  (Mean: 9.77, Std: 11.89)
Pow Block time statics ->  (Mean: 19.13, Std: 19.61)
Pos Block time statics ->  (Mean: 19.96, Std: 20.87)

Block time from blocks 5500 to 25000 (with 3 miners left):

Latest block number: 26796
Fetching data from block #5500 to #25000
Block time statics ->  (Mean: 9.89, Std: 11.45)
Pow Block time statics ->  (Mean: 19.38, Std: 18.90)
Pos Block time statics ->  (Mean: 20.18, Std: 21.43)

Block time overall:

Latest block number: 26787
Fetching data from block #500 to #25000
Block time statics ->  (Mean: 9.86, Std: 11.54)
Pow Block time statics ->  (Mean: 19.33, Std: 19.04)
Pos Block time statics ->  (Mean: 20.14, Std: 21.32)

aionnetwork / aion-unity-poc

Experiment over a long period #44