[BUG] Safe Balances Spell not capturing safes and forward-filling data

safeintern commented 2 months ago

Description

The Safes balances spells (Ethereum, Optimism, Polygon) are not properly capturing safes and forward-filling data from previous dates

Current behavior

As you can see in these queries (ethereum, optimism, polygon, the number of wallets for each day in these spells drop off greatly.

Expected behavior

The reality is that because the spell entails forward-filling the data from previous dates, there shouldn't be a decrease in number of wallets for each day.

Impacted model(s)

safe_ethereum.balances
safe_optimism.balances
safe_polygon.balances

Possible solution

None. Thank you!

safeintern commented 2 months ago

@jeff-dude can you please take a look 🙏🏻?

jeff-dude commented 2 months ago

my first thoughts are:

can you confirm the tokens balances table you read from represents all of your intended safes each day, as in does it look like it forward fills for you? since you join the two, it would of course filter out if not
incremental models and window functions can be difficult to get to work together properly. when you read each incremental run, you filter down the source data to a particular timeframe. its possible its only getting a subset of addresses due to this

jeff-dude commented 2 months ago

@0xRobin to compare to arbitrum RWA PR on balances

duneanalytics / spellbook