[WIP] Shard Chain v2

[hwwhww]

Shard Chain implementation of the current EIP docs. (Please ignore the previous version)

Description

1. sharding/main_chain.py

• MainChain inherits from ethereum.pow.chain.Chain, this object maintains top-level blocks and a list of ShardChain.
• New instance variables
  • shards: a list of ShardChain
  • shard_id_list: a set of shardIds that the validator keeps tracking of
• New functions
  • init_shard(self, shardId): initializes a new ShardChain object and add it to MainChain
  • add_shard(self, shard): adds an existing ShardChain object to MainChain
  • has_shard(self, shardId): checks if the validator is tracking of this shard
  • get_expected_period_number(self): gets default expected period number to be the next period number
  • get_period_start_prevhash(self, expected_period_number): gets period_start_prevhash by expected_period_number of collation header
  • update_head_collation_of_block(self, collation): after added a collation, updates ShardChain.head_collation_of_block field
    • TODO: haven’t been tested!
  • reorganize_head_collation(self, block, collation): after added a block, reorganizes head collation
    • TODO: haven’t been tested!
  • handle_orphan_collation(self, collation): handles the orphan collation (previously ignored collation)
• TODO After finished validator_manager_contract.add_header function, start to override MainChain.add_block function to:
  • [ ] Track on logs to receive collation headers
  • [ ] If the collation of this block hasn’t been downloaded, try to download it
  • [ ] Apply reorganize_head_collation function to set head collation

2. sharding/shard_chain.py

• ShardChain is a class for storing and maintaining the collations.
• Instance variables
  • shardId - the shard ID of the shard
  • collation_blockhash_lists[collation_header_hash] -> list[block_hash]: a map records which blocks contain a given collation header
  • head_collation_of_block[block_hash] -> head_collation: a map records at a given block, which collation is the head
• Functions
  • add_collation(self, collation, period_start_prevblock, handle_orphan_collation) - adds a collation to the shard chain
    • Since ShardChain is a member of MainChain, there are two additional input parameters: the block of collation.header.period_statr_prevhash (period_start_prevblock) and a callback function handle_orphan_collation for accessing block data
    • TODO: It seems weird to use callback function for this purpose? It may need to be refined or refactored.
  • get_parent(self, collation): gets the parent collation of a given collation
  • get_collation(self, collation_hash): gets the collation with a given collation hash
  • get_score(self, collation): get the score of a given collation
  • is_first_collation(self, collation): checks if the given collation is the first collation of this shard
  • get_head_collation(self, blockhash): gets head collation
    • TODO: haven’t been tested!
• TODO After finished validator_manager_contract.contract.add_header function, start to:
  • [ ] Test the get_head_collation function
  • [ ] Modify property head(self) (head collation)
  • [ ] Delete old junk data

3. sharding/collator.py

• create_collation(chain, shardId, parent_collation_hash, expected_period_number, coinbase, txqueue=None) - create a new collation
• apply_collation(state, collation, period_start_prevblock) - apply a collation to shard chain
• TODO
  • [x] Sign collation
  • [ ] Verify collation header
  • [ ] Handle gas

4. sharding/state_transition.py

This module contains the rewritten functions related to state transition from ethereum.commom, ethereum.meta andethereum.pow.consensus modules for sharding.

• mk_collation_from_prevstate(shard_chain, state, coinbase): make collation from previous state (refer to ethereum.common.mk_block_from_prevstate)
• add_transactions(state, collation, txqueue, min_gasprice=0): add transactions to a collation
• update_collation_env_variables(state, collation): updates collation variables into the state (refer to ethereum.common.add_transactions)
• set_execution_results(state, collation): sets state root, receipt root, etc (ethereum.pow.common.set_execution_results)
• validate_transaction_tree(collation): validates that the transaction list root is correct (ethereum.pow.common.validate_transaction_tree)
• verify_execution_results(state, collation): verifies the results by Merkle Proof (ethereum.pow.common.verify_execution_results)
• finalize(state, coinbase): applies rewards and commit (refer to ethereum.pow.consensus.finalize)

5. Other TODO

• [ ] Update sharding/tests/tools/tester to generate contracts on shard chain

Installation

For more details of environment, please refer to pyethereum.

sudo apt-get install libssl-dev build-essential automake pkg-config libtool libffi-dev libgmp-dev

git clone https://github.com/ethereum/sharding/

cd sharding

pip install -r requirements.txt

To install the dependencies necessary for development (testing, ...), run:

pip install -r dev_requirements.txt

How to run the tests

pytest sharding/tests/