Setup indexer for Spacewalk usage metrics

[annatekl]

Context

This ticket is for indexer setup in order to enable efficient retrieval of aggregated data over multiple blocks for Spacewalk.

Requirements:

• [ ] Set up the retrieval of:

Current chain state data:	Usage / Traction Metrics	Description	Timeframe
$ Amount Issued per Asset	Indicates the demand for specific assets within the system. If certain assets are experiencing high volume: introduce transaction fee, increase collateral.	total issuance up to the current time in USD
Total Number of Issue Transactions	Reflects the activity on the bridge. If low for certain assets, consider adding support for additional assets: marketing, pools to invest, collaboration with other projects.	week, month, all
Total Volume of Issues per asset	The total value of assets issued within a given period. Monitor to ensure there is enough liquidity available for users to facilitate transactions.	week, month, all in USD
Total Number of Redeem Transactions	Reflects the activity on the bridge. Reflects changes in user behavior, liquidity requirements. Action: patterns or trends analytics.	week, month, all
Total Volume of Redeems per asset	The total value of assets redeemed within a given period. Assess to manage associated risks, such as asset volatility or liquidity constraints.	week, month, all in USD
Total Collateral per asset	Total Collateral available per asset	Point in time

Health	Description
Collateralisation ratio per vault	For each vault, the current collateralisation ratio, which would suggest the health of the vault
Collateralisation ratio per asset	For each asset, the current collateralisation ratio, would suggestion the health of the bridge

• [ ] Output data on a page

[annatekl]

Hey team! Please add your planning poker estimate with Zenhub @b-yap @bogdanS98 @ebma @gianfra-t @TorstenStueber

[gianfra-t]

For some of these points I think is better to check the chain state directly like we were discussing in the slack. That doesn't mean it cannot be done in the indexer though, we can keep track of it also, it is just that the most reliable source for any current "snapshot" will always be the chain itself.

[TorstenStueber]

Yes, I also think that we should always directly use the chain state if this already gives sufficient information so that we can keep our indexers as simple as possible.

Such snapshot information is:

• Amount issued per asset
• Number of external vaults (the chain does not know what an external vault is but we can retrieve all vaults and then exclude our own ones by a script)

Possibly also the total collateral locked, however, there is some kind of aggregation script over all vaults required.

[ebma]

Ensure the indexer is configured to efficiently handle data retrieval from various blocks.

I don't understand this requirement. That's a basic feature of every block indexer, no?

---

For the rest

Amount issued per asset

Can be queried with "Chain State" > tokens::totalIssuance(currency)

Number of external vaults (if any)

As @TorstenStueber mentioned, internal vs external vaults is something that we make up and it doesn't make sense to implement this distinction into the indexer.

Total collateral locked by external vaults (if any)

Can be queried for all vaults with "Chain State" > vaultStaking::totalStake(0, None)

Total number of issue and redeem transactions

All issue and redeem requests can be queried with "Chain State" > issue::issueRequests(None) or redeem::redeemRequests(None)

From just the chain state it would be very difficult to tell these things for a specific timeframe though. So if you wanted to know 'How many XLM were issued from 01.12.23 - today?" we'd need to touch the indexer after all.

[gianfra-t]

The problem with total issuance from the state is that it will not be possible to see the accumulated right? Only the snapshot.

[ebma]

@gianfra-t you are referring to the total issuance subtracting the amount of redeemed assets again? That's true, but is it a problem? Do we want to know the total amount of minted tokens without incorporating the burned assets @pendulum-chain/product?

[gianfra-t]

Yes I was referring to that. So from the chain we get like you said the issue amount subtracted the redeemed. But perhaps another useful metric (to be defined of course) is the total accumulated bridged amount over time. And for that we need something like the squids I guess.

[ebma]

@annatekl can you move this ticket to the pendulum-squids repository instead?

[ebma]

I still don't think this ticket can be worked on though. As I pointed out in my previous comment, none of the items in the description necessarily require an indexer. As @gianfra-t pointed out, there are some metrics that we can only derive with an indexer. But it is not clear if we want to have those.

@pendulum-chain/product can you please make perfectly clear what the requirements are and why? Who is the consumer of this data? Is it just for internal use? Are external parties interested in this data? How do you define 'Amount issued per asset'? The amount that was issued ever (ignoring how many of those assets were burned again) or the amount that is currently issued?

[annatekl]

Please add your planning poker estimate with Zenhub @TorstenStueber

[annatekl]

the ticket description was updated

[TorstenStueber]

@ebma do you think that there are enough events emitted by Spacewalk pallets to infer all this information in the indexer – so that for this ticket only the indexer code needs to be touched?

[ebma]

Thanks for updating the description @pendulum-chain/product. A few questions:

• You say that the 'Total Volume of Issues' is denoted within a given period. What should that period be? Is it okay to have this available for periods of 24 hours or do you need it more/less fine-grained?
• The 'Total Number of Issue Transactions' is easily queriable from chain. Do you want to be able to look at this number for time periods again?
• The 'Total Volume of Issues/Redeems' does not mention the target currency. I assume you are interested in the USD-value of the total volume?

I want to point out that you have to be cautious when interpreting the troubleshooting metrics. What we can do is count the number of cancelled issue/redeem requests but since users have to manually cancel their redeem requests, they might do this way after the period for the redeem passed.

Thinking about the requirements overall, I guess that you want to be able to look at the development of each of them over time although it's only explicitly mentioned for the 'Total volume of issues/redeems'. Thus, I would propose we implement some snapshotting as is done with the Zenlink pair data e.g. here. In order to do this we need to know the expected granularity of the periods. @pendulum-chain/product Do you want to be able to analyze the changes over time per hour, day, week, or some other schedule?

do you think that there are enough events emitted by Spacewalk pallets to infer all this information in the indexer – so that for this ticket only the indexer code needs to be touched?

I think all the required data is in either the events or stored on-chain. For example, for deriving the USD amount issued per asset we will need to look at the DIA prices, but they are also already available at the indexer so it should not be too difficult.

[annatekl]

@ebma replying to your first question from Feb 1 - the user is internal but at some point we would like to have the data available to external user (similar view to interlay. For the Total Amount Issued: This interpretation considers the cumulative amount of the asset that has been issued over time, regardless of whether some of those assets have been burned or redeemed later.

[annatekl]

@ebma Total Volume of Issues 24 hours, Total Number of Issue Transactions/24 hour - is easier to read as the current on chain data plus with the plans to have the stats available to the external user it make sense to have it in a form that is easy to read.

[annatekl]

Total Volume of Issues/Redeems/currency - I thought about original currency @vadaynujra WDYT?

[vadaynujra]

@annatekl I think it's best to confirm with the stakeholders that will consume this data, as to what currency the volume should be denoted in.

[annatekl]

@vadaynujra thanks for your input. I think USD would work best for everyone, based on what we've talked about before.

[ebma]

Should we mark this as on-hold until we decided what approach we want to pursue for the Spacewalk analytics?

[annatekl]

@vadaynujra shall we close it as this will be covered by dune analytcis?

[vadaynujra]

Not yet @annatekl . Let's hear back from the Polkaholic team who is helping us set it up, see if it meets our requirements as intended, and then close this with comment - Covered via public Dune dashboard.

[prayagd]

Because KPIs covered by the Public dune dashboard https://dune.com/substrate/pendulum