dipdup-io / dipdup

Modular framework for creating selective indexers and featureful backends for dapps
https://dipdup.io
MIT License
97 stars 52 forks source link

Configurable size of handler batch #1091

Open droserasprout opened 2 months ago

droserasprout commented 2 months ago

A batch of matched handlers passed to batch handler currently always equals one block. It's called in _process_level_data. The task is to make it configurable, either by number of blocks or matched handlers.

NueloSE commented 1 month ago

I am applying to this issue via OnlyDust platform.

My background and how it can be leveraged

i am developer with experience using python and some python frame work

How I plan on tackling this issue

Identify Current Batch Logic:

Locate where the batch handler processes one block at a time in _process_level_data. Make Batch Size Configurable:

Add configuration options for batch size, allowing users to set by number of blocks or matched handlers. Modify _process_level_data:

Update the function to check the configuration and process either a specific number of blocks or matched handlers in each batch. Test the New Configuration:

Ensure the system correctly handles different batch sizes and configurations without breaking the current functionality. Document Changes:

Update documentation to explain the new configurable batch options.

droserasprout commented 1 month ago

Unassigning for inactivity.

Mystic-Nayy commented 1 month ago

@droserasprout can I be assign to work on this?