All opinionated data, e.g. data that makes a strategy dependent should be moved into the crawl path

neume-network / strategies

Indexing strategies for the neume network.

GNU General Public License v3.0

11 stars 7 forks source link

All opinionated data, e.g. data that makes a strategy dependent should be moved into the crawl path #253

Closed TimDaub closed 1 year ago

TimDaub commented 2 years ago

e.g. we're defining a filtering function here in the code for the upstream source file that soundxyz-call-tokenuri expects: https://github.com/neume-network/strategies/blob/c1be9dfb034863e778252140ad8c273d22b8a123/src/strategies/soundxyz-call-tokenuri/extractor.mjs#L19
but it'd be better if this function was defined as an argument in the crawl path as to not make the strategy overly dependent on the outcome of the call-block-logs or logs-to-subgraph results

/cc @sirnicolaz as he mentioned this /cc @il3ven because you might be interested in implementing this

il3ven commented 2 years ago

I think we can remove the strategy-factory pattern if we pass arguments in crawl path. On the plus side, it will reduce the number of strategies.

il3ven commented 2 years ago

I think we can remove the strategy-factory pattern if we pass arguments in crawl path.

I was wrong. It isn't as simple as passing the filterFunc from crawl path. Suppose we have a single strategy call-tokenuri instead of soundxyz-call-tokenuri and zora-call-tokenuri which we can call twice and pass filterFunc. But the results will be written to the same file and the upcoming strategies such as soundxyz-get-tokenuri will have to filter from it.

Taking a step back, I don't understand why filterFunc needs to be added to the crawl path. filterFunc alone will not make the strategy generic. We have other props also such as signature.

TimDaub commented 2 years ago

But the results will be written to the same file and the upcoming strategies such as soundxyz-get-tokenuri will have to filter from it.

haha funny that you came to the same conclusions independently because I just created that issue: https://github.com/neume-network/strategies/issues/258

Taking a step back, I don't understand why filterFunc needs to be added to the crawl path. filterFunc alone will not make the strategy generic. We have other props also such as signature.

fair. e.g. here would be a blueprint that filters events using the call-block-logs strategy: https://github.com/neume-network/blueprints/blob/main/src/filter-mint-in-singleton-music-nft-contracts/step.mjs

il3ven commented 1 year ago

is this issue still valid? because blueprints were getting out of hand and we don't want to import libraries like eth-fun into crawl path.

TimDaub commented 1 year ago

I've wrote down some defining qualities here: https://github.com/neume-network/blueprints#differences-blueprints-vs-strategies

TimDaub commented 1 year ago

I'll be none the less difficult to understand what should be a blueprint or a strategy. Actually let me close this until we found a better problem to work on.