estuary / flow

🌊 Continuously synchronize the systems where your data lives, to the systems where you _want_ it to live, with Estuary Flow. 🌊
https://estuary.dev
Other
594 stars 47 forks source link

Bigsum reduction #1476

Closed ligustah closed 4 months ago

ligustah commented 4 months ago

Description:

Our use case requires summing up big numbers (mainly unsigned 256 bit integers), which is not currently supported.

Workflow steps:

Use like this:

{ "reduce": { "strategy": "bigSum" } }

Inputs need to be string-formatted integers or numbers. I excluded floats for now, mainly because there were some funky precision errors and we don't need them to work.

Documentation links affected:

(list any documentation links that you created, or existing ones that you've identified as needing updates, along with a brief description)

Notes for reviewers:

This is very much a draft. I haven't previously contributed to this project and I'm fairly new to Rust, so this is just my quick attempt at making it work. It is heavily inspired by the sum strategy, just using the most downloaded big decimal library under the hood. Happy to hear any feedback on this implementation.


This change is Reviewable