[Feature][Bug?] Keep table intact on config/schema removal

ponder-sh / ponder

A backend framework for crypto apps

https://ponder.sh

MIT License

613 stars 95 forks source link

[Feature][Bug?] Keep table intact on config/schema removal #841

Open exp-table opened 5 months ago

exp-table commented 5 months ago

Let's start with a specific example ;

You have one ponder that's polyvalent (you add/remove contracts and events regularly). You only need events from block A to block B for contract X. Once it has done its job, you remove altgother the concerned code in index.ts, config.ts and schema.ts. Relaunching ponder will result in the previously populated table to be removed entirely.

It would be nice to have an option in the config perhaps to choose to maintain the table.

0xOlias commented 5 months ago

Hi, welcome and thanks for opening.

This pattern is somewhat orthogonal to the way Ponder works. Specifically, Ponder aims to be deterministic, reproducible, and fast enough such that we can avoid maintaining user state between hot reloads / redeployments.

Have you considered:

Rather than remove a contract from ponder.config.ts, set the endBlock field? Ponder will stop indexing the contract at that height.
Create multiple Ponder apps?

Keen to help - would be great to hear more details about your use case.

exp-table commented 4 months ago

Our use case is essentially tracking on-chain actions from users, which validates "quests" we have with partnering projects.

We are also using supabase, so not railway's db system, so every I/O has a cost! Repopulating tables from scratch everytime ramps these up, and also confuse users sometime because we are using data from tables to display informations!

We have thought about having one ponder app per quest, but the issue is then the limitation on how many pools we are allowed on supabase and how many one ponder needs (default is 30).

Though I will admit this is a very particular use case, deviating from the essence of ponder!

0xOlias commented 4 months ago

Thanks for sharing, this is very helpful context. Don't have an immediate solution, but we will keep this use case in mind moving forward.

I will note that we're about to ship a release that will significantly speed up reindexing speed when using a remote database. After this change, it might be fast enough that you can tolerate leaving all quests around.