search

gridcoin-community / Gridcoin-Tasks

Gridcoin community tasks repository
https://gridcoin.us
MIT License
24 stars 5 forks source link

investigate why 2 superblocks created in short time (frequency: 20%) #122

Closed skcin closed 6 years ago

skcin commented 6 years ago

Issue by Erkan-Yilmaz Sunday Apr 23, 2017 at 11:37 GMT Originally opened as https://github.com/Erkan-Yilmaz/Gridcoin-tasks/issues/117

For April now this happened for like 20% of all superblocks (and for the last 5 SBs: 60%)

Here we'll collect the happenings so it can be analyzed more (A):

April 22:

April 18:

April 17:

April 4:

March 31:

March 21:

(A) not complete list yet (for April it should be complete), if you have more details to above, please comment. thx

skcin commented 6 years ago

Comment by Foggyx420 Sunday Apr 30, 2017 at 10:08 GMT

From my looking back at all these superblock issues I've noticed some important facts that stand out that tell me a lot.

1) This happens within a short time frame 2) Happens by a user staking both blocks 3) The users that have staked these double blocks have had extensive balances and stake numerous times through out the day (double digits in blocks) 4) The consensus hash and superblock hash which must be the same are identical in each of the superblocks.

This is not really a bug at all as the wallet is behaving as it should and is programed in my opinion.

From the code It says the nodes will not accept a superblock if the hash is not the popular one. The hashes are correct in both at the time so they were accepted!

The popular neural hash only updates every 10 blocks from my understanding as the neural network works as a team and request everyone in neural network to come to a quorum and popularity changes and hashes as well. So if the user with a high balance supporting the neural network does manage to stake more then one block in that time frame while having the popular hash then bingo they are the proud owner of 2 superblocks instead of 1. They come through as valid as they should. If the same user happened to stake 2 blocks in that same period but wasn't the popular consensus hash they would just be two staked blocks in a small window and we would all look the other way. <- true story!

Maybe question for this is -> Is the superblock considered a superblock while its not yet received the required confirmations to be a valid block? (20 confirms I believe before my wallet adjusts to superblock). With that question in place once there is a 'good' superblock the network sleeps for 12 hours so the code says. But again the question -> is it good when its completely confirmed or just from staked? If it is after its got the required confirms it does explain this happening as well as it is not officially confirmed.

We all know our wallet does not stop attempting to stake when there is staking weight to be staked. High balance users will stake numerous blocks a day thou there is a chance you can stake the next block right after as you always have a chance to stake a block. We see this here and there and even as gridcoinstats.eu shows it with nice red field.

For example well will call a guy named V and you all know him! (LOL) and he is a rich guy haha. he has 8.6 million GRC and over 1400 magnitude that can be seen from his stats. He stakes numerous blocks a day. We have all seen it users staking blocks sometimes 2 in a row or two within a 10 block range. These users tend to have very high balances thus they have more weight staking on the network. He staked 2 superblocks back to back. So within that 10 blocks neural network consensus while having the popular hash at the time he got the superblocks.

I have not seen anything about how the superblock is really chosen but from what I can guess and figure is to be the super block you have to have the popular hash/consensus hash at the time you stake a block in a time frame the network is doing what it does.

In connect block you can see that it will reject a new superblock if its not the popular hash as well if the consensus hash and superblock hash do not match and if the consensus hash and legacy neural hash don't match it will reject as well. From that the two back to back superblocks passed through connect block without any issues as it was designed to let correct superblocks go through.

I've been looking over the code over last while and I don't see any issues in way its behaving generally in my opinion but I am no expert.

My thoughts:

It is not broken, however as humans we want to make everything work perfectly. The users staked the blocks fair and square and matched up with neural network with popular hash. They had high balances which enabled them to stake a lot more then the average user to begin with. With getting to stake 2 blocks in a short period of time within the 10 blocks while having the popular hash they won the prize; two superblocks. All the double superblocks are within that 10 block range. every 10 blocks when neural network is working as a team it has to come to a quorom and the hashes and popularity change as well as the network is updated (syncing there too?). Timing is everything right?

So to conclude they have the highest balances and support the neural network and stake more often during a time period where neural network attempts to come to consensus. They have the highest chance to stake the superblocks as is and odds increase of 2 superblocks in that 10 block range with those factors behind them. You don't see two different users staking superblocks within that 10 block range as they can't both have popular hash during that time frame. the users in neural network with lower balances have a lot smaller of a chance of staking the superblock as they would need to stake the block while having popular hash. Big balance users in neural network have more chances in the game. The wallet performed how it is programmed in my opinion.

Can it be fixed? Yes but I don't believe it is something to even remotely worry about. Both superblocks were valid. When all this began along time ago I don't think the consideration of 8.6 million GRC against a neural network could cause anything. Rob closed this yes and I agree it is not an issue. It is almost a boasting bonus.

What could be done? 1) a check point (locally?):

Comments? I'm all ears! criticism? I'm all ears. It's good and healthy for the gridcoin network to hear everyones take on this so we can maybe be like the neural network?? Lets come to a consensus!

Id like to find out how many users with beacons versus how many users working in neural network. and how many in neural network are high rollers.

skcin commented 6 years ago

Comment by tomasbrod Sunday Apr 30, 2017 at 12:02 GMT

Only negative is that double superblock wastes around 20kB in our block-chain. Rewriting the NN to work cross-platform will help the network and I am really excited for the process. Whad do you think about restriction so that one CPID can't stake more than one block within a hour?

skcin commented 6 years ago

Comment by Foggyx420 Sunday Apr 30, 2017 at 19:11 GMT

@tomasbrod I don't see an issue with a lower balance client with the 1 hour limit. But higher balance users it could maybe be like a bottleneck and reduce the block count for them per day? perhaps maybe it could be that you can't stake a block for 10 blocks from previous block if the last one was a superblock? if we went that route. I don't think more then a superblock scenario should be considered in that if anything is changed.

Anyway I wanted to post the blocks I had checked. The blocks not shown here from main OP is likely because the double superblocks did occur but one was rejected.

Date: April 4th Blocks: 862017/862012 Blocks apart: 5 Same user: YES, Same GRC address Whom: Investor Type: PoS Notes: grc address owned by an expired beacon. They have a high balance 2.8 Million GRC and the superblock hash/neural hash was the same in both blocks

Date: March 31st Blocks: 857929/857928 Blocks Apart: 1 Same user: YES Whom: Vortac Type: DPoR Notes: same super hash/neural hash in both blocks. They have a high balance 8.6 Million GRC.

Date: Match 21st Blocks: 849746/849742 Blocks apart: 4 Same user: YES Whom: Thermophys Type: DPoR Notes: same super hash/neural hash in both blocks. They have a high balance 3.5 Million GRC.

Date: March 17th Blocks: 846234/846232 Blocks apart: 2 Same user: YES Whom: Vortac Type: DPoR Notes: same super has/neural hash in both blocks. They have a high balance 8.6 Million GRC.

skcin commented 6 years ago

Comment by grctest Sunday Apr 30, 2017 at 19:28 GMT

Whad do you think about restriction so that one CPID can't stake more than one block within a hour?

This would potentially degrade network health, as individual CPIDs would be limited to producing 24 blocks max per day, several users stake more frequently than this due to their balances as it is. Perhaps we could enforce a limit on producing SBs after producing one, but that isn't solving the root problem.

skcin commented 6 years ago

Comment by tomasbrod Sunday Apr 30, 2017 at 19:44 GMT

So Investors can stake a superblock? OK, no problem with that. In AcceptBlock function the second superblock should be rejected. The fact that there are two superblock close to each other does not mean any harm, it may be a side-effect of bigger, yet undiscovered, security flaw. What is the root problem @grctest?

skcin commented 6 years ago

Comment by Foggyx420 Sunday Apr 30, 2017 at 21:46 GMT

i don't think its a problem. way i see it is the 10 block window with 2 stakes in it and the popular hash is correct so they both superblocks. The network is not wrong as the consensus is there. if we made neural network come to qourom every block that would just add congestion?. I believe this is just something that was not for seen.

The checks in check block only checks if new superblock doesn't match the consensus hash at the time and rejects it.

Two reasonable things I see is to:

1) Leave it alone! and once neural network is ported this should be a less occurrence. It hasn't caused any damage and hasn't given the stakers bonus points. 2) Add an additional check that says if they staked a superblock in last 10 blocks and both consensus hashes are the same then the 2nd block that would normally be a superblock as well becomes a normal block with no appending of the binary superblock. It's not a huge change as a superblock is just a staked block with the superblock binary information, etc in it.

I think option 2 is the least problem causing if a fix is really absolutely wanted.

@tomasbrod is accept block a good option?

I'm thinking that the stake should not be rejected after all it is a stake. Maybe this could be done in the create block general areas? Say if a user has a staked a superblock within past 10 blocks and consensus hash is the same between the two blocks then just create a normal PoS/DPoR block instead.??

I'm trying to think of the less impact method and I don't think rejecting a block is a great idea or fair. if we can implement something before the block is created with checkpoints that prevent it from becoming a second superblock I think that would be more of a winner.

I'm no expert but learning more and more as I go everyday.

Another thing I forgot to mention is it does seem to happen also when the previous pending superblock has not yet reach the required confirms to be fully accepted. your mag does not change immediately after the superblock but does once it gets the required confirms of 10

skcin commented 6 years ago

Comment by tomasbrod Friday Jun 02, 2017 at 13:50 GMT

Today superblock #915288 was staked after 44 hours since previous superblock. And 6 blocks after that, another superblock was staked by the same user. Note that this two superblocks have different neural** /superblock hash.

{
"Block #915294" : "1bf2097593c1065a005f7df9fdc591f25430dbfc9965ba8fb8b41e5a5cf0c355",
"Date" : "06-02-2017 01:16:48",
"Average Mag" : 87.70737389,
"Wallet Version" : "v3.5.8.8-g-research",
"Neural Hash" : "796d3af6f55e0d8330a4dccf8a5f7753"
"CPIDv2" : "b59c71815e3d8190ddf63e71da57ea52",
},
{
"Block #915288" : "df7e1959ba159c79294cd17cb34f43442fbf31d0f729224d39f6878cd9bc11e0",
"Date" : "06-02-2017 01:07:44",
"Average Mag" : 81.75777002,
"Wallet Version" : "v3.5.8.9-g-research"
"Neural Hash" : "084638a6686a11ab2ec3a3c87b5a1b3f"
"CPIDv2" : "b59c71815e3d8190ddf63e71da57ea52",
},
{
"Block #913641" : "cfda6d953f7aaff1da72b446aca2172976ee6fbfd7bdc042fafdb98d0c2ee150",
"Date" : "05-31-2017 05:07:44",
"Average Mag" : 87.02055287,
"Wallet Version" : "v3.5.8.9-g-research"
"Neural Hash" : "11e9bb799ca97ff241d9608356e4b12a"
"CPIDv2" : "b59c71815e3d8190ddf63e71da57ea52",
},
skcin commented 6 years ago

Comment by tomasbrod Friday Jun 02, 2017 at 14:12 GMT

This would potentially degrade network health, as individual CPIDs would be limited to producing 24 blocks max per day, several users stake more frequently than this due to their balances as it is.

That is unfortunate @grctest . There already is a limit that CPID stakes must be at least one hour apart. But this limit is in effect after 10 blocks from last stake. Other limits: payment_age > 60*60, payment_age > Difficulty*144, coin_age > 4*60*60 and coin_age > Magnitude+Boost.

skcin commented 6 years ago

Comment by Foggyx420 Friday Jun 02, 2017 at 16:42 GMT

The two superblocks were by two differnet users.

One grc address is linked to a cpid while the other is not on gridcoinstats. We can't use the whole cpid when it comes to them being back in investor mode as its the same cpid. Also another fact is both staked by two different versions of the wallet. .8 and .9. To top that off the neural popular hash had already changed. after the 10th block meaning XXXXX0 so when the second one staked it had the popular neural hash at the time however the previous superblock had not had the require confirms to be a valid block for the wallets to use it as a superblock. Takes 10 i believe before even my wallet updates my mag,etc. and then it remains pending for the rest of the 100 blocks like a standard PoS/DPoR.

I still believe the problem is the 10 blocks confirm before the network says hey we have a superblock i just haven't found where that is.

And as for the 60 * 60 payment_age is a kinda cheap. Its basically hey you staked a block now lets cripple your staking for an hour. Kinda found it interesting that it was commented out for telling a user that its the reason they are not staking lol

skcin commented 6 years ago

Comment by Erkan-Yilmaz Wednesday Aug 23, 2017 at 07:59 GMT

see https://github.com/gridcoin/Gridcoin-Research/issues/534

skcin commented 6 years ago

Comment by Foggyx420 Wednesday Aug 23, 2017 at 08:09 GMT

looks like same issue. the 4sbs from today are all from same user.

WhiteCat6142 commented 6 years ago

1105121&1105122 too and both are created by same miner

tomasbrod commented 6 years ago

Thanks for reporting that this is still happening. IIRC there is a clientside check in 36.3.0. Definitely network rule was added in 36.3.17.

In @jring-o's article State of the Network Report #3 12-9-17 Suggests that double superblocks are still happening.

Average Time Between Superblocks: 17.4302 Hours

The later of the superblocks wasb6ca3b2775cdd2a166a38631685886b2df9022b61385404d965777d11f45f84f. Both were created by @bgb with v3.6.3.0-unk. He should post logs here.

Pythonix commented 6 years ago

Can this issue be closed? Many updates on the NN have been done.