search

subsquid / gitcoin-hackathon-issues

DeFi & Cross-chain Interoperability Hackathon
4 stars 0 forks source link

Aggregate data from multiple chains in a single Subsquid API #2

Open dzhelezov opened 2 years ago

dzhelezov commented 2 years ago

Collect data from multiple chains in a single Subsquid API

Prize Bounty

Bounties Totalling 7000 USDC ($6000 for the winning submission and $1000 for the runner-up).

Challenge Description

Subsquid/Hydra is a toolkit for building and deploying powerful and fully customizable GraphQL APIs for blockchain data within minutes. The developer has full control over the API and data shape (defined in schema.graphql) and the way the data is transformed from raw on-chain transactions (defined in mappings scripts).

At the moment the mappings handle only events emitted by a single Substrate-based chain, fetching them from a dedicated endpoint API which supports efficient batching (such as this for Kusama).

The goal of this bounty is to add the possibility to fetch events from multiple data sources defined in the manifest. Since there is no canonical order you should come up with an ad-hoc synchronization algorithm to make sure that the events coming from different chains are merged in a deterministic and sound way. For example, one possible solution would be to look at the event timestamps. The more creative, tamper-proof, and verifiable design is implemented, the better.

Submission Requirements

A Submission should be a fully runnable Subsquid project with some sample mappings, likely a fork of https://github.com/subsquid/hydra-template with a custom libs substituting @subsquid/hydra-processor and @subsquid/hydra-common and possibly @subsquid/hydra-indexer. You are free to modify the format of manifest.yml as you want (you will likely add a section where the Ethereum events of interest are described and probably more). You are free to use any external services like Infura.

The submission should clearly explain the strategy of pulling the data and document the new features of the manifest file if introduced. When deployed the resulting GraphQL API should expose data from multiple chains in some way.

Judging Criteria

Winner Announcement Date

A winner will be announced shortly after the hackathon is complete.

Resources

Where to get help

gitcoinbot commented 2 years ago

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

This issue now has a funding of 7000.0 USDC (7000.0 USD @ $1.0/USDC) attached to it.

xiaods commented 2 years ago

@dzhelezov please clear what multiple chains need support.

dzhelezov commented 2 years ago

@dzhelezov please clear what multiple chains need support.

Any two independent chains will do. However, since Subsquid already supports events emitted by a single Substrate-based chain, I believe extending that to multiple Substrate chains (say, Kusama and Karura) will be the most natural (and easiest) thing to do.

In particular, instead of sourcing the events from a single indexer, one can try to extend the processor code and fetch data from two independent indexers simultaneously l.

gitcoinbot commented 2 years ago

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

Work has been started.

These users each claimed they can complete the work by 7 months, 3 weeks ago. Please review their action plans below:

1) cpurta has started work.

Scoped out a majority of the work that needs to be done in order to get multiple chains into a single API. To allow users to query a specific chain they can specify an enumerated chain that the API supports through the GraphQL schema. The plan is to get a hydra-typegen fork to support specifying multiple metadata sources and generate the needed TypeScript interfaces for the indexers. Some work will need to be done to get the indexers to pull the correct types for each chain they are indexing. Finally, the last piece will be to get the GraphQL schema updated with an Enum type per chain that the API supports and get the queries updated to support a specified chain, and update the resolver(s) to pull the indexed chain's data. 2) ivanmolto has started work.

I will aggregate data from multiple chains in a single Subsquid API

Learn more on the Gitcoin Issue Details page.

xiaods commented 2 years ago

I believe extending that to multiple Substrate chains (say, Kusama and Karura) will be the most natural (and easiest) thing to do.

so the first target bingo is support Kusama and Karura? are you some prefer chain recommend

xiaods commented 2 years ago

update status:

  1. create project setup: https://github.com/xiaods/hydra-one
  2. run README steps, sync data from substrate chain data.
xiaods commented 2 years ago

@dzhelezov I follow the subsquid project's README to setup local devel environment. I have a question: Does it need to be synchronized with the Substrate master chain？it will take long time to syn for me.

xiaods commented 2 years ago

when the sync on work, but the grahsql sever not work. even the console show the server is up to now https://localhost:4000/grap is not work. do you know what reason.

dzhelezov commented 2 years ago

try http://localist:4000/console. Note that https is not enabled by default

dzhelezov commented 2 years ago

@dzhelezov I follow the subsquid project's README to setup local devel environment. I have a question:

Does it need to be synchronized with the Substrate master chain？it will take long time to syn for me.

You don't have to wait until it's fully synced, the API is going to work straight away. You may also find useful going though the following video: https://www.youtube.com/watch?v=i1A5ApN98YY&list=PLvTrX8LNPbPn-t6ON3wscq50i6zGtt-dk&index=2

xiaods commented 2 years ago

@dzhelezov I follow the subsquid project's README to setup local devel environment. I have a question: Does it need to be synchronized with the Substrate master chain？it will take long time to syn for me.

You don't have to wait until it's fully synced, the API is going to work straight away. You may also find useful going though the following video: https://www.youtube.com/watch?v=i1A5ApN98YY&list=PLvTrX8LNPbPn-t6ON3wscq50i6zGtt-dk&index=2

thanks for your help. it works.

gitcoinbot commented 2 years ago

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

Work for 7000.0 USDC (7000.00 USD @ $1.0/USDC) has been submitted by: