Open dirmansebastian opened 6 years ago
my eth light geth node is syncing and it's over block 6305955
I have never tried pushing blocks using light geth blocks. I will try it and update this issue.
but, if i do ether_sql scrape_block --block_number 6305955 it also connect to DefaultSettings(infura, not my api ) and it works
To use your personal infura setting use this command:
ether_sql --settings='PersonalInfuraSettings' scrape_block_range --start_block_number 6305955
PS: Thanks a lot for stres testing the library PPS: refrain from posting your private infura key on a public forum like this.
that's how i've used it: ether_sql --settings='PersonalInfuraSettings' scrape_block_range --start_block_number 6307820
, but no luck
anytime, i'm glad i can help a little it's for test, no worry, if people are lazy to create an account on infura they can use it
Strange, I tried the command and it works for me:
$ ether_sql --settings=PersonalInfuraSettings scrape_block --block_number=6307820
[2018-09-10 22:14:30,229][INFO][ether_sql.session] Connected to the db ether_sql
[2018-09-10 22:14:30,231][INFO][ether_sql.session] Connected to Infura node
[2018-09-10 22:14:30,231][INFO][ether_sql.globals] Pushing the session PersonalInfuraSettings in local thread
[2018-09-10 22:15:01,746][INFO][ether_sql.tasks.scrapper] Commiting block: 6307820 to sql
More, stranger thing is that the above command works for you when using DefaultSettings
but does not work with PersonalInfuraSettings
Can you add the line LOG_LEVEL = "DEBUG"
in your class PersonalInfuraSettings
to see where is the process actually being killed, and copy paste the output?
(envname) [20:45 ubuntu@ip-172-26-15-152 ether_sql] > ether_sql --settings='PersonalInfuraSettings' scrape_block_range --start_block_number 6307821
[2018-09-10 20:45:16,099][DEBUG][ether_sql.session] DEBUG
[2018-09-10 20:45:16,164][INFO][ether_sql.session] Connected to the db ether_sql
[2018-09-10 20:45:16,165][INFO][ether_sql.session] Connected to Infura node
[2018-09-10 20:45:16,165][INFO][ether_sql.globals] Pushing the session PersonalInfuraSettings in local thread
[2018-09-10 20:45:16,224][DEBUG][web3.RequestManager] Making request. Method: eth_blockNumber
[2018-09-10 20:45:16,225][DEBUG][web3.providers.HTTPProvider] Making request HTTP. URI: https://mainnet.infura.io/HcekpL4KhjR4L0pCa4o8, Method: eth_blockNumber
[2018-09-10 20:45:16,230][DEBUG][urllib3.connectionpool] Starting new HTTPS connection (1): mainnet.infura.io:443
[2018-09-10 20:45:16,617][DEBUG][urllib3.connectionpool] https://mainnet.infura.io:443 "POST /HcekpL4KhjR4L0pCa4o8 HTTP/1.1" 200 44
[2018-09-10 20:45:16,617][DEBUG][web3.providers.HTTPProvider] Getting response HTTP. URI: https://mainnet.infura.io/HcekpL4KhjR4L0pCa4o8, Method: eth_blockNumber, Response: {'jsonrpc': '2.0', 'id': 0, 'result': '0x604106'}
[2018-09-10 20:45:16,618][DEBUG][ether_sql.cli] 6308102
[2018-09-10 20:45:16,618][DEBUG][ether_sql.cli] 6307821
[2018-09-10 20:45:16,618][DEBUG][ether_sql.models.blocks] 6305957
Killed
Response: {'jsonrpc': '2.0', 'id': 0, 'result': '0x604106'}
, there should be 'id':1 right?
I think I got the reason why the process is being killed. The command first checks for missing blocks in the sql i.e. number of blocks which are missing between start_block_number and block 0, this is a heavy task and might be the reason for Killed
command.
[2018-09-10 20:45:16,618][DEBUG][ether_sql.models.blocks] 6305957
This log is returned at line below and the process is being killed after that. https://github.com/analyseether/ether_sql/blob/0ce3a74c7ce802d6db721b7066826dd873b98f07/ether_sql/models/blocks.py#L111
Try running the above command with setting --no-fill_gaps
$ ether_sql --settings=PersonalInfuraSettings scrape_block_range --start_block_number=6000000 --no-fill_gaps
Response: {'jsonrpc': '2.0', 'id': 0, 'result': '0x604106'}, there should be 'id':1 right?
Yes the id should be 1 but we are not using the id in any way it to kill our process.
yes, u're right, i got it now. --no-fill_gaps work as workaround with infura
running same command with Geth: ether_sql --settings='PersonalGethSettings' scrape_block_range --start_block_number 6308345 --no-fill_gaps
(envname) [21:50 ubuntu@ip-172-26-15-152 ether_sql] > ether_sql --settings=PersonalGethSettings scrape_block_range --start_block_number=6000048 --no-fill_gaps
[2018-09-10 21:51:13,389][INFO][ether_sql.session] Connected to the db ether_sql
[2018-09-10 21:51:13,390][INFO][ether_sql.session] Connected to Geth node
[2018-09-10 21:51:13,390][INFO][ether_sql.globals] Pushing the session PersonalGethSettings in local thread
Traceback (most recent call last):
File "/home/ubuntu/ether_sql/envname/bin/ether_sql", line 11, in <module>
load_entry_point('ether-sql', 'console_scripts', 'ether_sql')()
File "/home/ubuntu/ether_sql/envname/lib/python3.6/site-packages/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/home/ubuntu/ether_sql/envname/lib/python3.6/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/home/ubuntu/ether_sql/envname/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/ubuntu/ether_sql/envname/lib/python3.6/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/ubuntu/ether_sql/envname/lib/python3.6/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/home/ubuntu/ether_sql/envname/lib/python3.6/site-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "/home/ubuntu/ether_sql/ether_sql/cli/__init__.py", line 91, in scrape_block_range
mode=mode)
File "/home/ubuntu/ether_sql/ether_sql/tasks/scrapper.py", line 66, in scrape_blocks
add_block_number(block_number)
File "/home/ubuntu/ether_sql/envname/lib/python3.6/site-packages/celery/local.py", line 193, in __call__
return self._get_current_object()(*a, **kw)
File "/home/ubuntu/ether_sql/envname/lib/python3.6/site-packages/celery/app/task.py", line 379, in __call__
return self.run(*args, **kwargs)
File "/home/ubuntu/ether_sql/ether_sql/tasks/scrapper.py", line 143, in add_block_number
timestamp=iso_timestamp)
File "/home/ubuntu/ether_sql/ether_sql/models/receipts.py", line 60, in add_receipt
status = bool(to_int(receipt_data['status']))
TypeError: 'NoneType' object is not subscriptable
same command should work with geth or is not yet enabled?
Describe the bug killed
To Reproduce run
ether_sql --settings='PersonalGethSettings' scrape_block_range --start_block_number 6305955
, my eth light geth node is syncing and it's over block 6305955but, i get killed when i try to push data in database
i get killed too if i use same command with infura:
ether_sql --settings='PersonalInfuraSettings' scrape_block_range --start_block_number 6305955
i've modified settings.py like that:
but, if i do
ether_sql scrape_block --block_number 6305955
it also connect to DefaultSettings(infura, not my api ) and it worksExpected behavior to not get killed and to push blocks into database
Desktop (please complete the following information):