search

brianleect / etherscan-labels

Full label data dump of top EVM chains in JSON/CSV.
MIT License
249 stars 73 forks source link

Automation for generating the file on a nightly basis? #12

Open travisbotello opened 1 year ago

travisbotello commented 1 year ago

Hey there! Many thanks for this awesome script. Is it somehow possible auto generate those files every night and add it to this github repo? For example like: https://github.com/sapics/ip-location-db

brianleect commented 1 year ago

Hmm I'm not too sure tbh since there's heavy reliance on logging in via etherscan first using selenium.

Main challenge would be

Alternative I suppose could be to leave an instance logged in running and simply scrape everything from the last pages of each recorded column.

Not sure about the compute costs though and perhaps a soln that relies on attaching cookies https://github.com/brianleect/etherscan-labels/issues/9 would be much more efficient or a completely separate library used to controlling browser scraping might be better?