Extract uncles of a transaction

[beeshot]

What are uncles?

https://blog.ethereum.org/2016/10/31/uncle-rate-transaction-fee-analysis/

Is uncle rate an indicator for centralization?

[kiview]

This blog is, in general, a very reliable and high quality information source for Ethereum.

[beeshot]

[kiview]

Is this a screenshot from bloxberg? As I understood uncles (valid but stale blocks, rejected over time because of eventual finality mechanism in PoW consensus), there shouldn't be any in bloxberg.

If there are, I wonder what they mean. Bugs in AuRa or Parity? Hinting at network and node failures?

[beeshot]

Yes, it is a screenshot from bloxberg. Thought it may be interesting to dig deeper, so i posted it here to remember the connection. I found it there and didn't hear about of it before. I didn't get into it too deep. But from what I understood, it is a way to reintegrate slated blocks, and miners will get up to 75% of the original reward. Trying to prevent centralization and well-connected mining pools. However, it also talks about the uncle rate and the likeliness of a block being rewarded by the amount of transaction it contains (refering to Blogpost). I was also looking for it in the Mastering Etherum book, but didnt find anything about it. So yeah, good question what this means here.

[kiview]

There are no miners in Bloxberg. We are using Authority Round consensus, leading to sealing blocks instead of mining them. So on the one hand, incentivization does not come into play, on the other hand, this should not occur since AuRa consensus has a fixed cadence for authority nodes that participate and seal blocks.

So if we see those, I think this hints at some misbehaviour in the network, wherever this comes from. Could be bugs, failures, maybe clock being out of sync? We should investigate Uncle rate and I'd hope it would be extremely low, else I think the network is not very reliable.

What needs to be done from a technical side then is:

• Identify blocks as uncles (how can we identify this?)
• Calculate uncle rate over transactions

[beeshot]

It is a reference from a paper with the following title: The Impact of Uncle Rewards on Selfish Mining in Ethereum https://bitslog.com/2016/04/28/uncle-mining-an-ethereum-consensus-protocol-flaw/

[beeshot]

Extract Uncle Rate:

https://blog.ethereum.org/2016/10/31/uncle-rate-transaction-fee-analysis/

Here is a script to generate some source data: http://github.com/ethereum/research/tree/master/uncle_regressions/block_datadump_generator.py

Here is the source data: http://github.com/ethereum/research/tree/master/uncle_regressions/block_datadump.csv

The columns, in order, represent block number, number of uncles in the block, the total uncle reward, the total gas consumed by uncles, the number of transactions in the block, the gas consumed by the block, the length of the block in bytes, and the length of the block in bytes excluding zero bytes.

We can then use this script to analyze it: http://github.com/ethereum/research/tree/master/uncle_regressions/base_regression.py

The results are as follows. In general, the uncle rate is consistently around 0.06 to 0.08, and the average gas consumed per block is around 100000 to 300000. Because we have the gas consumed of both blocks and uncles, we run a linear regression to estimate of how much 1 unit of gas adds to the probability that a given block will be an uncle. The coefficients turn out to be as follows: ..... more on blog

[kiview]

When interacting with bloxberg web3j gives us already uncle data, so we just need to store it. This can be seen in the web3j logs, logging the HTTP response.

[kiview]

I rephrased the title, since determining the rate is a second processing step after getting the raw data.

[Mschnuff]

added extraction of uncles per transaction (inherited from the corresponding block). this is the output for the first 200 blocks. no uncles so far.

[{"timestamp":1547107485,"fromAddress":"0xab59a1ea1ac9af9f77518b9b4ad80942ade35088","toAddress":null,"uncles":[]},{"timestamp":1547107625,"fromAddress":"0xab59a1ea1ac9af9f77518b9b4ad80942ade35088","toAddress":null,"uncles":[]},{"timestamp":1547107680,"fromAddress":"0xab59a1ea1ac9af9f77518b9b4ad80942ade35088","toAddress":"0x9850711951a84ef8a2a31a7868d0dca34b0661ca","uncles":[]}]

[Mschnuff]

on second thought, doing it this way, doesnt make a lot of sense because there might be uncles for blocks without transactions.

[Mschnuff]

work in progress feel free to take a look: https://github.com/internet-sicherheit/ethereum-cache-creator/pull/37

[kiview]

There are not uncles in bloxberg, as far as we see. If anyone should see an uncle somewhere, we can reinvestigate.