ZcashFoundation / GrantProposals-2018Q2

Submission site for 2018Q2 Zcash Foundation grant proposals.
26 stars 2 forks source link

Open source zcash blockchain analysis platform (development and hosting) #39

Open rmhere opened 6 years ago

rmhere commented 6 years ago

Blockchain Exploration Research Group (BERG) Wrocław University of Science and Technology, Wrocław, Poland https://www.ii.pwr.edu.pl/~michalski/berg/

The proposal covers creating and hosting an open source zcash blockchain analysis platform for research purposes.

In early 2018 we provisioned a tool for Bitcoin (authored by M. Pieczka, a member of BERG) that is providing the researchers with the ability to query and filter the block database in order to perform further analyses, such as:

This tool is open source (https://github.com/MarcinPieczka/Mongo-BTC-Blocks-Database/) and it consists from two main components: the query engine for querying the blockchain and extensible API that is now being extended with new services to perform multiple analyses in-site instead of just sending the data outside the server for further research.

The goal of the project proposal is to extend and update the tool to support zcash too to enable the research on zcash blockchain. This tool will be also open source plus it will be made available as a free service for other researchers hosted by us for one year after the completion of the project (hopefully longer).

The project team will consist of three BERG members: PI Dr. Radosław Michalski, Marcin Pieczka, and one more programmer. The project is estimated to last for about three months (5 person months).

tromer commented 6 years ago

@rmhere, can you provide examples of use cases for your analysis tool?

Also, can you comment on how it compares/relates to the tool developed under https://github.com/ZcashFoundation/GrantProposals-2017Q4/issues/24 (see their draft report, https://cryptolux.org/images/d/d9/Zcash.pdf)?

rmhere commented 6 years ago

Thanks @tromer for your comment.

Firstly, the tool is intended to provide generic information about blocks mined so far. By doing so it provides information for researchers interested in studying different aspects of the cryptocurrency (economy, security, network science etc.) - the obtained results can be of interest of multiple stakeholders – researchers per se, blockchain developers, market analysts and so on. As we are a group that is mostly interested in looking at blockchain as at a complex network, nowadays we use the platform we developed for the purposes of analyses we provided information about in the grant proposal (we can describe the details of analyses we did so far if you want us to do so, just write). However, the idea is that the extensible API should also provide the ability to run on-site analyses, depending on the researchers’ goals in order to support reproducible research as well as let the researchers to focus on their key work, not on writing the same code multiple times.

The analyses we think one could perform using the platform migrated to zcash + with extended API: • discovery of critical phenomena in the zcash blockchain • multivariate study of the evolution of the network over time • automatic classification of addresses’ types in order to find out whether the anonymity of the node can be preserved • assets’ transfer analyses (trends in the volume of transactions)

Moreover, as we wrote, we would be happy to host the platform for the scientific and developers community for a year after finishing the project + it will be open source (as the current platform for Bitcoin blockchain is). In our group, we also have a number of researchers and the platform will be also extended by the services supporting our own research on blockchain.

Comparing to the platform mentioned in https://github.com/ZcashFoundation/GrantProposals-2017Q4/issues/24, some of the goals are similar, yet we believe we could bring some more network science background with our work (as we focus on network science). Also maybe there is a possibility to merge the efforts (we are open for collaboration). Apart from that, we offer hosting of the platform as mentioned in the proposal.

If you have some more questions, please feel free to ask.

tromer commented 6 years ago

Can you help us understand how network science applies here, and give concrete examples of interesting analysis methods and API features that exceed those provided by https://github.com/ZcashFoundation/GrantProposals-2017Q4/issues/24 (as well as standard block explorers API, the Insight software, etc.)?

rmhere commented 6 years ago

@tromer, let me then add some more details

Firstly, as Insight or standard block explorers, the Bitcoin version of the tool does the same in terms of basic functionality (but please read further :). Namely, it parses the blocks into more suitable form for queries and further analyses. However, as we progress with the development of our Bitcoin version, we see that in order to perform more sophisticated analyses, transferring blocks' and transactions' information to end-users is absolutely not the way to go, since before running the analysis a lot of data has to be transferred. This is why we believe that this sort of tools should provide their extensive API to be available to run analyses on-site and just send the results. E.g., there is no need to send some blocks to compute shortest paths or send transactions information to calculate the flow of cryptocurrency. Our intention is to build a number of API services we already use in our research and invite others to suggest (or even develop) more. The goal is to make the platform a point you query for information instead of a tool you query for blocks/transactions.

Secondly, I read the description and report of https://github.com/ZcashFoundation/GrantProposals-2017Q4/issues/24. How do I see the differences between this proposal and the one above? This proposal will go in the following directions:

We would like to start with extending our API with basic measures and clustering algorithms, as we already did some research on that, then we wish to go to towards node classification algorithms.

Trying to answer your question on network science. This is a discipline derived from graph theory you could think of as an applied graph theory. It is observed that in a number of applications network science methods improve the quality of results as they look at the network structure, network dynamics, nodes' positions in graphs etc., something that is often overlooked. By doing so, network science brings new quality into disciplines like biology, chemistry, sociology etc. When thinking of using the benefits of network analysis to blockchain, I believe this could lead to an identification of the trends in network evolution, network robustness, link prediction, node identification, cluster analyses and so on. Moreover, we could answer the questions on which models is the zcash network most similar to and others. I believe you already know or looked at some articles about network science, but if not, this is the easiest way: https://en.wikipedia.org/wiki/Network_science. Of course, we could not cover all in this project proposal, so we will focus on what was listed above.

tromer commented 6 years ago

The Zcash Foundation Grant Review committee has reviewed your pre-proposal, including the above discussion, to evaluate its potential and competitiveness relative to other proposals. Every pre-proposal was evaluated by at least 3 (and typically more than 4) committee members .

The committee's opinion is that your pre-proposal is a promising candidate funding in this round, and the committee therefore invites you to submit a full proposal. Please submit a full proposal by June 15th, following the detailed structure described in the Call for Proposals. We encourage you to submit a draft as early as possible, to allow for community feedback.

rmhere commented 6 years ago

Thank you for reviewing the pre-proposal and we were really happy to be invited to the full proposals stage. Our full proposal is therefore attached below. In case of any more questions, please do not hesitate to ask.

zcash_grant_proposal_39_upd.pdf

(updated with small corrections in the text, not influencing the merits of the proposal)

tromer commented 5 years ago

@rmhere, we are about to finalize the grant funding recommendations. Are there any updates in your status or plans that we should be aware of?

rmhere commented 5 years ago

@tromer, I queried both planned collaborators and nothing changed, we await the decision

sonyamann commented 5 years ago

I'm thrilled to inform you that the Grant Review Committee and the Zcash Foundation Board of Directors have approved your proposal, pending a final compliance review. Congratulations, and thank you for the excellent submission!

Next steps: Please email josh@z.cash.foundation from an email address that will be a suitable point of contact going forward. We plan to proceed with disbursements following a final confirmation that your grant is within the strictures of our 501(c)(3) status, and that our payment to you will comply with the relevant United States regulations.

We also wish to remind you of the requirement for monthly progress updates to the Foundation’s general mailing list, as noted in the call for proposals.

Before the end of this week, the Zcash Foundation plans to publish a blog post announcing grant winners to the public at large, including a lightly edited version of the Grant Review Committee’s comments on your project. The verbatim original text of the comments can be found below.

Congratulations again!

Grant Review Committee comments:

Proposes an on-line platform for analyzing Zcash transactions, to help researchers investigate the Zcash on-chain data effectively and and reproducibly.

Even after t-addresses are deprecated, lots of statistical information can still be extracted from the blockchain. Having this tool available will enable researchers, for example, find parameters that are estimable and comparable with other currencies.

The following reservation was raised. There have already been 3 papers showing the flaws with users interacting with T-Z addresses (e.g., https://blog.z.cash/maintaining-privacy/). It is crucial for the proposers to focus on facilitiating novel discoveries, and to strike for useful information even with the eventual deprecation t-addresses. For example, consider how network-layer information can be captured and conveyed by the tool.

We recommend funding this proposal.