search

balancer / dune-spellbook

SQL views for Dune Analytics
Other
7 stars 12 forks source link

Organize spells into folders #83

Closed viniabussafi closed 3 weeks ago

viniabussafi commented 1 month ago

Our spells are getting quite numerous and will increase in v3, so I propose we break them down into folders, in order to keep them more organized. Structure would be the following, using the same structure already on spellbook for the _sectors folder:

balancer --> spell category --> chain

mendesfabio commented 1 month ago

it would be great to have a better structure there. did you find any team doing something similar? i'm just wondering if that doesn't break dbt and/or the structure already in place. also what does spell category mean? liquidity, volume, etc?

viniabussafi commented 1 month ago

I'm using this as reference: https://github.com/duneanalytics/spellbook/tree/main/models/_sector

My initial division would be something like: balances bpt flashloans liquidity pools protocol_fee support trades vebal

You can find it here: https://github.com/balancer/dune-spellbook/tree/organize-folders/models/balancer

mendesfabio commented 1 month ago

got it but that's for sectors not projects. I meant if there's a model/project structured this way or another - that's the folder that dbt compile will look for. I assume dbt compile will just loop through the nested folders and it should work but worth double-checking it.

Also another trade-off to keep in mind is you don't have reference for the columns in some of the new schema yaml files and that will lead to errors - for instance:

https://github.com/balancer/dune-spellbook/blob/organize-folders/models/balancer/pools/balancer_schema.yml#L14

I'm mostly wondering if this doesn't break anything and/or whether Dune team will approve it.

viniabussafi commented 1 month ago

Got it. I asked them about it on a PR I have open and let's see what they say. Also, I plan on doing a bit of improving on schema files, fixing some wrong/missing descriptions and a few typos along the way

viniabussafi commented 1 month ago

Seems to work as expected: view it on GitHub Since I haven't gotten a reply from the dune team, I'll go ahead and open up a PR

viniabussafi commented 1 month ago

PR open

viniabussafi commented 1 month ago

PR should pass tests now. This PR does not introduce any change regarding the spells per se, other than their location. Except for legacy trades spells, which I took the liberty of removing All schema files have been renamed as _schema and improved regarding syntax, formatting and column descriptions. I have validated outputs and they stay the same, as expected.

viniabussafi commented 3 weeks ago

PR merged