hwwhww commented 6 years ago

Make pyethapp alive again

Although this PR is supposed to be split into many PRs, but our integration testing needs all commits to make pyethapp alive again. :'(

1. Fixed Python2/3 compatibility

Based on @changwu-tw ’s PR #266 From #266:

Use future package
Fix bytes/str problem
Modify to py3 style code
Modify import _socket to import socket

And added:

Using ethereum.utils.to_string and ethereum.utils.is_string to make sure string is bytes
Using pyethapp.utils.MinType and pyethapp.utils.to_comparable_logs to support dict comparison and sorting (which is not allowed in Python3)

2. Use unicode for json-rpc encoding

For compatibility, use unicode for both Python2/3 encoding. Not sure if it would impact on any other applications?

3. Updated travis, tox and setup.py for testing both Python2.7 and Python3.6

So the travis would help us to test both Python2.7 and Python3.6.

Due to the dependency on latest pyethereum source code, I have to modify the requirements.txt:

- ethereum>=1.5.1
+ https://github.com/ethereum/pyethereum/tarball/develop

It could be changed back after the latest pyethereum is uploaded to PyPI.

4. Added `process_time_queue` gevent to call `chain.process_time_queue` periodically

Simple mechanism: implemented it by using BaseService gevent event and hacked ChainService
Maybe it could be improved to a TimeQueueManager service in other PR
Default checking period is 5 sec as geth
Related PR: https://github.com/ethereum/pyethereum/pull/816

5. Trigger `mine_head_candidate()` again after `block.mining_hash != mining_hash`

For a single node, the miner doesn’t take any measure to handle the case block.mining_hash != mining_hash before, so I fixed it by adding a gevent.

def recv_found_nonce(self, bin_nonce, mixhash, mining_hash):
        log.info('nonce found: {}'.format(encode_hex(mining_hash)))
        block = self.chain.head_candidate
        if block.mining_hash != mining_hash:
            log.warn('mining_hash does not match')
+           gevent.spawn_later(0.5, self.mine_head_candidate)
            return False
        block.header.mixhash = mixhash
        block.header.nonce = bin_nonce
        if self.chain.add_mined_block(block):
            log.debug('mined block %d (%s) added to chain' % (
                block.number, encode_hex(block.hash[:8])))
            return True
        else:
            log.debug('failed to add mined block %d (%s) to chain' % (
                block.number, encode_hex(block.hash[:8])))
            return False

6. Fixed `on_receive_getblockheaders` and added tests

Referencing the algorithm from geth (https://github.com/ethereum/go-ethereum/blob/10181b57a9fb648f5fd424ca611820a3cf42c42b/les/handler.go#L375-L454)
How to verify it:
```
pytest pyethapp/tests/test_eth_service.py::test_query_headers
```
7. Enable using custom network_id in config.yaml
Replace this field if network_id is set in config.yaml

How to verify them all

I made a sample development environment with docker-compose: https://github.com/hwwhww/docker-pyeth-dev

If anyone wants to try or review this PR, please change the pyethapp branch manually. :)

coveralls commented 6 years ago

Coverage increased (+0.6%) to 49.93% when pulling cd6e700cb6fb64a8465e3a98077c77e0ced7e2cb on hwwhww:pr_py3_dev_env into fe206b475c0683c5ed853c0280098e7b43426b8d on ethereum:develop.

coveralls commented 6 years ago

Changes Unknown when pulling 42f1326f47aba1a2e7ba7fd73a3973e38ccd9fc0 on hwwhww:pr_py3_dev_env into on ethereum:develop.

ethereum / pyethapp