worldcoin / signup-sequencer

signup-sequencer repository
MIT License
67 stars 33 forks source link

Added batching using DB. #724

Closed piohei closed 3 months ago

piohei commented 3 months ago

This is a first step to introduce batches and transactions table. This is not HA solution yet - just the first step towards it.

Motivation

Make small changes instead of one big.

Solution

Now processing batches is changed into 2 steps. First one is to create a batch (select identities, leaf indexes, etc.). Then the batch is saved in database. Second step is reading data from database (batch) and executes it. Proper batch execution includes storing transaction id in database.

As an alternative solution we could save in database a batch with a proves (no need to calculate it later). It will allow second step (transaction creation/execution) to not use any tree at all. It will require more memory used in database (it can be cleared from time to time as we don't need all the batches forever) but will make HA solution much easier.

PR Checklist