volsa / etherface

Ethereum Signature Database
https://www.etherface.io
GNU General Public License v3.0
171 stars 22 forks source link

Offline lookups #21

Open zerjioang opened 1 year ago

zerjioang commented 1 year ago

I would like to have a way to run a massive lookup of unverified contracts against this database. However, I think that making it against the online version might impact both application's performance, your server, and my script. For this reason, I would like to propose creating a wikipedia style dump (https://dumps.wikimedia.org/backup-index.html) in which 4bytes information database can be downloaded for offline usage.

volsa commented 1 year ago

I might release a signature dump in the coming days, in the meantime feel free to use the API. I'm sure it can handle your load, though you might want to cache known signatures; see for example https://github.com/uibk-ethpector/ethpector/blob/467428db2254c5baf027374316f49c273bc94a85/src/ethpector/data/signatures.py#L55-L63

cgebe commented 1 year ago

Hey, would also be interested in a dump. We need to check signatures for all transactions block by block (live every 12 seconds). Is this load okay for now? Of course we would cache / store signatures across blocks. Just for building up the database on our own, it would be great to kickstart with a dump. csv hex, signature would totally suffice.

volsa commented 1 year ago

I uploaded a dump on GitHub, had to split them into two parts though because of GitHub's maximum file size restriction. Head over to https://github.com/volsa/etherface/tree/master/signature-dump to download them. Also keep the following two issues in mind

cgebe commented 1 year ago

Thank you very much. That will makes things a lot quicker!