search

aragon / nest

A grants program to support the development of the ecosystem
https://aragon.org/project/grants
Creative Commons Zero v1.0 Universal
140 stars 78 forks source link

Analyse Ether --> Open data analytics framework for ethereum #42

Closed ankitchiplunkar closed 6 years ago

ankitchiplunkar commented 6 years ago

Aragon Nest Proposal: Open data analytics framework for ethereum

Abstract

The current ethereum blockchain has almost 6 Million blocks, performing data-analysis on this data-set will provide useful insights on decentralized exchanges, tokens, ENS, prediction markets etc. The ability to quickly, cheaply perform data-analysis will be very useful to increase the overall knowledge of whole community and increase the understanding on inner workings of several smart contracts. Moreover such a framework will be very useful for developers and private chain users to analyze their chains.

There are two main tasks when creating this framework.

  1. Getting the past and current data from blockchain. There are currently three main methods to extract data from ethereum network:

    1. Personal node: Running a ethereum node, which with the current block-size needs a SSD hard disk and a couple of weeks to sync in archive mode.
    2. Infura nodes: Provides access to basic JSON RPC methods and does not give access to the full state of the network some time in the past.
    3. Etherscan API's: Closed source API's to get data from ethereum node but again the ancient data is not available easily.

  2. Creating a frontend library to run queries on the data and generate visulizations.

We propose to build Analyse Ether to facilitate the analysis of ethereum data. By providing a backend library to convert ethereum data in queryable format and a browser based frontend library to perform real-time data-analysis of ethereum transactions.

Deliverables and development timeline

  1. ether_sql (1-2 months): A back-end library to stream ether data to an sql database.

    • Support to add extra tables based on contract calls
    • Support to decode ethereum bytecode

  2. explorer (1-2 months): An open source block explorer written in python flask, which uses ether_sql database.

    • Expose ancient and real-time data through API's

  3. analyse_ether (3-4 months): A frontend library to handle sql queries, visualizations, and interact with the ether_sql database.

    • Support to create sql queries
    • Support to create basic visualizations
    • Support to create Jupyter notebooks

Grant size

Funding: $120k in ETH, $20k each month for 6 months. Split into chunks paid out over achieved deliverables.

Success reward: Up to $50k in ANT, given out when all deliverables are ready.

Application requirements

Development time-line

The development timeline will be the following one in regards to each deliverable:

  1. ether_sql: July 2018
  2. explorer: September 2018
  3. analyse_ether: November 2018

Note

We have a MVP running initial version of deliverable 3, to get access to the MVP fill out this form. As we complete each deliverable we will also host these services on our servers as well.

ghiliweld commented 6 years ago

Not part of the team here but I'm really interested in seeing how this goes, it would really help my project out.

mariapao commented 6 years ago

Hi @ankitchiplunkar thanks a lot for submitting your proposal. I just requested access to the MVP.

ankitchiplunkar commented 6 years ago

Hi, @mariapao thanks a lot for your interest, I have just sent you an invite to checkout the MVP.

bingen commented 6 years ago

Hi @ankitchiplunkar , can I have access too? Thanks.

mariapao commented 6 years ago

Hi @ankitchiplunkar we have been reviewing the documentation and playing around with the MVP and we like it. However, we think this is project is very similar to a project we already funded: gnarly. Do you guys know about it? Gnarly is a community driven project so it might be a good idea to reach out to the team and discuss about what you guys are doing. Let me know if you want me to make an intro. Also, regarding the block explorer, there is an open block explorer NOW initiative. There are several projects doing similar things. Have you guys heard about it?

For the reasons above, we have decided not to approve this proposal at the moment and focus the resources on other unattended but necessary tools. Thank you again for the work work you put into the application, we really appreciate it!

ankitchiplunkar commented 6 years ago

Hi @mariapao, thanks for thoroughly reviewing the application.

Upon your suggestion we have reviewed Gnarly and agree that design wise, Gnarly and ether_sql are similar (ethereum node --> postgresql) but the final objectives are different. It would be great if you can make an intro with the Gnarly team.

Moreover we recently came in contact with open block explorer NOW initiative and took part in their 6th call. Here is the demo of our MVP during the call https://youtu.be/A78opcM7veI?t=6m51s

bingen commented 6 years ago

Yes, I agree the final objectives are different, but I guess it would be nice to have a common library that converts from ethereum node to sql that anybody can use. Not sure, maybe it's too late for this if you both have it already implemented.

Anyway, I have been playing around with the MVP and I really liked it, you did a very good job!

ankitchiplunkar commented 6 years ago

@bingen thanks a lot for the reviews :smile:

You will continue having access to the MVP and can always run analysis. Do tell us if you have any recommendations about the product.

PS: In the upcoming weeks we will also add state data in the platform.

mariapao commented 6 years ago

Hi @ankitchiplunkar can I have your email? Thanks :)

ankitchiplunkar commented 6 years ago

sure, its: ankit@analyseether.com