italanta / kujali

Financial Planning, Operations & Reporting
Other
2 stars 0 forks source link

⚡ 🧮 Calculate budget headers (which hold budget result table) on save #14

Open jrosseel opened 1 year ago

jrosseel commented 1 year ago

On save of a budget, the budget header needs to be calculated. This is a technical requirement to allow for optimal calculation of the budgets, minding time vs storage considerations.

Explanation

The "Budget Header" is a shorthand and denormalised object which contains the value of the result table for a given budget. It has the following structure: { id: ${unix of Date.now()}, name: 'Budget name', bId: 'budget id}, startY: 2022, duration: 5, header: [[100,100,100, ...], [..., [... ]] }

Header is stored as a bi-dimensional array with for each of the years in which the budget is active an array of the calculated result values for each month of that year.

The Budget Header is to be stored on the path "/orgs/{orgId}/budgets/{budgetId}/headers/{timestamp}".

Every time the budget is saved, a new record is added to the collection with id is latest timestamp.

The function calculating the header has to run on a background function _bdgtCalculateHeader which has two registrars:

_bdgtCalculateHeaderOnSaveBudget _bdgtCalculateHeaderPubSub

_bdgtCalculateHeader expects the Budget object as input.

Regressive property of the procedure

It's important to note that the procedure has a regressive property. Indeed, if the budget is a child of a parent budget, the parent budget's header will also be affected once the child's header is calculated. For each parent, the PubSub functionality regressively calls _bdgtCalculateHeaderPubSub with the Budgets of each parent.

Distribution risk The regressive nature of this procedure . The function should therefore transaction lock onto the path "/orgs/{orgId}/budgets/{budgetId}/headers/{timestamp}" - The lock is ordered descending by timestamp and limit(1).

If the transaction lock is violated, implement exponential back-offs and limited retries.

Endless looping risk If the data is corrupted and a case exists where the budget hierarchy has a loop, this function will endlessly loop resulting into very expensive use of funds. Preempt this scenario by doing a graph walk before going into regression (walk all parents to ensure you don't come across yourself in the parent list. Terminate with error if you come across yourself).

Another way to do this, is to attach a stacked list to each of the regressions which holds all the budget headers part of the regression. The current budgetId is pushed onto the stack before the command is created and the _bdgtCalculateHeaderPubSub regression is called.

IanOdhiambo9 commented 1 year ago

Firebase does not support nested arrays as described above for the header property. I'm proposing a data structure like the one below (for a budget that starts in 2022 and runs for 3 years):

header : { 2022 : [100, 100, 100 .....dec] 2023 : [100, 100, 100 .....dec] 2024 : [100, 100, 100 .....dec] }

instead of

header : [[100, 100, 100 .....dec], [100, 100, 100 .....dec], [100, 100, 100 .....dec]]

@jrosseel