Python scripts for ETL (extract, transform and load) jobs for Ethereum blocks, transactions, ERC20 / ERC721 tokens, transfers, receipts, logs, contracts, internal transactions. Data is available in Google BigQuery https://goo.gl/oY5BCQ
docker run --net="host" ethereum-etl:latest stream --provider-uri http://host.docker.internal:8545 -e block,transaction,log,token_transfer,contract,token --output=postgresql+pg8000://postgres:password@host.docker.internal:5432/ethereum
2022-05-08 15:15:39,484 - root [INFO] - Using http://host.docker.internal:8545
Traceback (most recent call last):
File "/usr/local/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/local/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "ethereumetl/__main__.py", line 26, in <module>
cli()
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1128, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/ethereum-etl/ethereumetl/cli/stream.py", line 71, in stream
item_exporter=create_item_exporters(output),
File "/ethereum-etl/ethereumetl/streaming/item_exporter_creator.py", line 30, in create_item_exporters
item_exporters = [create_item_exporter(output) for output in split_outputs]
File "/ethereum-etl/ethereumetl/streaming/item_exporter_creator.py", line 30, in <listcomp>
item_exporters = [create_item_exporter(output) for output in split_outputs]
File "/ethereum-etl/ethereumetl/streaming/item_exporter_creator.py", line 73, in create_item_exporter
ListFieldItemConverter('topics', 'topic', fill=4)])
File "/ethereum-etl/blockchainetl/jobs/exporters/postgres_item_exporter.py", line 38, in __init__
self.engine = self.create_engine()
File "/ethereum-etl/blockchainetl/jobs/exporters/postgres_item_exporter.py", line 58, in create_engine
engine = create_engine(self.connection_url, echo=self.print_sql, pool_recycle=3600)
File "<string>", line 2, in create_engine
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/deprecations.py", line 298, in warned
return fn(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/create.py", line 559, in create_engine
dialect = dialect_cls(**dialect_args)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/dialects/postgresql/pg8000.py", line 303, in __init__
raise NotImplementedError("pg8000 1.16.6 or greater is required")
NotImplementedError: pg8000 1.16.6 or greater is required```