#bug; gaps in timeseries

[melotik]

Describe the bug Currently our timeseries data has gaps in it. It is usually the market snapshots. If there is no activity in a market in a given day we do not create the snapshot. Originally we did this to not show any stale data.

Expected behavior Going forward we want to remove any gaps in the timeseries. Options and potential downsides:

• As we support SDKs for protocol types we can include a pricing class that the dev must implement for the given protocol.
• Once this is implemented we can then accurately create market snapshots on all markets each time there is any activity in the protocol. This will fill in some of the gaps with non-stale data.
• However, it is still possible for there to be gaps in the data if there is no activity in the entire protocol. In these cases we can create all missing snapshots in retrospect. The issue is that we will have stale data for those snapshots.

[melotik]

Another question I want to pose, when there are still gaps in snapshots: Do we want to fill in stale data or do we want to let the user make the assumptions when they process the data? There are 3ish ways you can go about the retrospective snapshot creations:

1. Use the newest data from the latest snapshot
2. Average the data from the last/latest snapshot
3. Use the data from the last snapshot before the gap

[melotik]

There are 2 solutions to fix it, both have drawbacks:

1. If there is no activity in a protocol on a given day should we retrospectively create snapshots that have stale data. Personally I don’t think we should have stale data in subgraphs and if that is the case we should not create a snapshot.

2. We watch an active event that we know for sure will execute x blocks and we can use it to ensure we have fresh data even when there is no activity in the protocol. This will slow down syncing. Depending on how active the event is will determine how significant.