search

kamu-data / kamu-contrib

Place to share your kamu datasets
5 stars 0 forks source link

Explicit watermarks in RETH datasets #23

Open sergiimk opened 2 months ago

sergiimk commented 2 months ago

Rocketpool's tokens-minted dataset have not been producing new data points as token was saturated. The problem is that when data is not flowing we also don't update the watermark.

This ticket is to update the datasets to write explicit watermarks sourced from latest block's timestamps. Containerized ingest protocol should be updated accordingly to support this if needed.

sergiimk commented 1 month ago

Original description was:

Rocketpool had a big Houston upgrade in June and since then tokens-minted dataset stopped producing any data points.

We should investigate how to support the new version.

Investigation confirmed that this is valid - no new tokens have actually been minted due to RETH deposit pool being saturated. RETH is currently being traded on Balancer and Uniswap, but not currently minted.

The problem here is that our current datasets don't update watermark when data is not flowing. We should rectify this by writing watermark sourced from the latest block's timestamp. Updating the ticket description accordingly.