Budget Accumulation Issues

shmcgrath commented 7 years ago

When using the budgeting option (either yearly, monthly, etc.) Fava does not seem to calculate the budget until there is at least one transaction in the Expenses. An example from my files is below:

The budget line in my Beancount file reads like this:

2017-01-01 custom "budget" Expenses:Tech:Hardware "yearly" 750.00 USD

I expect the program to allocate 750.00 USD for the year. It is fine (excellent, actually) that this does it per day. The problem is that the budgeting does not seem to allocate properly until there is a transaction in that category. This is problematic for large, annual or semi-annual expenses. (I don't pay my car insurance until September) I would expect my budget to be accumulating from the start date (i.e. January 01).

Now for my example: The transaction that kicks off my Hardware category is $94.99 on February 21. When I look at the weekly balances, the balance for that week in the budget is a total of $15.97. That is not correct.

If I add a dummy transaction of $0.01 for January 2nd, the remaining budget drops to $9.79 for that week, which is closer to the correct value. If I look at the budgeting daily, the Jan 02 dummy transaction for $0.01 leaves me with a budget of $2.04.

A couple of clarifying questions:

If I am kicking off my annual budget on January 01, shouldn't the January 02 budget have $4.09 in it?
- Note: If I add another dummy transaction for $0.01 on Jan 01, the budget then properly allocates on Jan 02 ($4.09).
Why does the weekly budget fluctuate so much with and without a dummy transaction?
Is there something that I am doing wrong in my files? (There probably is)

Let me know if you have any more questions or if I should be asking this elsewhere. Thank you for all of the hard work on Fava, it is awesome!

aumayr commented 7 years ago

Sorry for the late response - I did not have time to look into this yet.

yagebu commented 7 years ago

Thanks for the detailed report! I'm currently travelling so I've only had time to really look at this now. I don't use budgets myself and so I don't test this part of Fava's functionality in my daily usage - I've found the following while trying to reproduce this issue.

When the balance for an account is empty, the budget wasn't displayed - this should be fixed now. Otherwise I think these problems come from the reporting intervals that Fava considered. If no time filter is set, it will only consider dates between the first and last transaction, so since the budgeting is computed daily, this might lead to wrong budgets. Can you try setting the time filter to 2017 to see if these issues persist? (I couldn't reproduce this issue with the filter set) If that fixes it, I guess the complete fix will be to also consider the first and last budget for the reporting interval.

shmcgrath commented 7 years ago

I will check on your suggested solutions this week. Sorry this has taken me this long. School has been overwhelming.

TomJohnZ commented 7 years ago

Thanks for the improvement, @yagebu! I use the budgeting feature quite regularly and find it to be one of the most useful tools in Fava. The new implementation is quite nice; it now correctly shows a budget even if no transactions have been posted to the account during the display interval.

However, it looks like a small regression was introduced; while the budget now currectly accumulates daily, it never decreases when transactions are posted. That is, using shmcgrath's example above, posting a transaction of $1.00 on Jan 1, the budget would still read $4.09 instead of $3.09 for that day. I think I found the bug and submitted a pull request, though you might want to check my change to balance_children.currencies (I don't fully understand the difference, but it corrected the problem for me).

And regarding the reporting intervals, I can confirm what shmcgrath is describing; as you described, budgets are only based off the currently selected reporting interval. I usually get around this by zeroing my budgets at the end of each month (transfering any credits/debts to a slush fund), but having it consider the first and last budget for the full reporting interval would be ideal (though it might introduce a bit of overhead if the budget period is long?).

aumayr commented 7 years ago

I use the budgeting feature quite regularly and find it to be one of the most useful tools in Fava.

@TomJohnZ Do you have any suggestions (changes, new features) how to make it even more useful?

TomJohnZ commented 7 years ago

Hmm, one feature I couldn't find was the ability to close a budget (like an account). It's easy enough to comment-out the budget line when it's no longer needed, but sometimes it's nice to go back and see if an account was over/under-budget in the past, even if it's no longer being tracked.

Other that that, the only part I originally found a bit confusing (somewhat related to shmcgrath's point) was that budgets are calculated based on the reporting interval, not from when the budget was opened. I think it would be convenient to have an option to change this behavior: I have some long-running budgets, and rather than show more than a year of transactions, I prefer to filter by month. However, I could see this possibly introducing more overhead, so I'm not sure how feasible it is.

If any of these seem worthwhile, I wouldn't mind trying to lend a hand. You guys already do such great work on this project, I feel bad dropping more requests on you all.

yagebu commented 7 years ago

Hmm, one feature I couldn't find was the ability to close a budget (like an account)

You can set the budget to zero, which should have the effect of "closing" the budget. Or doesn't that do enough?

Other that that, the only part I originally found a bit confusing (somewhat related to shmcgrath's point) was that budgets are calculated based on the reporting interval, not from when the budget was opened.

I don't see how it would be possible to change that. Say you have a yearly budget that started in the middle of last year and now you want to see a monthly report of this year. You can't substract the monthly account balances from a yearly budget so you have to convert them somehow. AFAICS the only way that works consistently for all intervals is breaking everything down into days as Fava does.

If you have a suggestion on how it should work differently, could you give some small example of how it could work?

TomJohnZ commented 7 years ago

Thanks for the input!

You can set the budget to zero, which should have the effect of "closing" the budget. Or doesn't that do enough?

This works somewhat, though any postings to the account after the "closing" appear as going over the (zero) budget. It's certainly not a major issue at all though.

AFAICS the only way that works consistently for all intervals is breaking everything down into days as Fava does.

I completely agree, breaking everything down into days is the best way to do it. Let me try to rephrase my suggestion:

Using your example, say we started a budget in the middle of last year, and now we want to see a monthy report of this year (for concreteness, say only March of this year). Currently, monthly account balances are subtracted starting from March 1st. However, if I filtered instead for Feb and March, the balances are subtracted starting from Feb 1st.

The fact that the budget was calculated based on my time filter initially confused me; it seemed more natural to have the balances be subtracted beginning at the start of the budget (halfway through last year) and only display the budgets for the dates I filtered. This way, the budget on any particular day is independent of the display interval I chose.

However, I now realize this still isn't optimal; if someone were to start a budget, and then at a later date change the budget, choosing when to start subtracting balances is ambiguous.

Like I'd said, I've found perfectly satisfactory workarounds for my use case, so these suggestions can easily be ignored as well. I'm sure you all have far more pressing changes you want to implement!

ngoonee commented 7 years ago

Since @aumayr asked for budget suggestions, one possibility is to have an optional upward-flow of budget amounts. Here's a portion of my budget currently:-

2017-01-01 custom "budget" Expenses:Food "yearly" 18500 MYR 2017-01-01 custom "budget" Expenses:Food:EatingOut "yearly" 12000 MYR 2017-01-01 custom "budget" Expenses:Food:Groceries "yearly" 5000 MYR 2017-01-01 custom "budget" Expenses:Food:Other "yearly" 1500 MYR

As you can see, the Expenses:Food account has an amount which is just the sum of all the amounts in the child accounts. This could be done automatically (probably with an option as perhaps some wouldn't like this).

Upshot of this is that, in the default fava view, we can see the 'big picture' (how much I've spent in this time period compared to my totals of all budgets) and then drill down to whatever level of specificity we want (in the above, sometimes I just want to see if I can still spend more on food this month). The reason for it being optional would be that sometimes the larger category (Food in this instance) may have a budget which differs from a simple sum, especially if the user tends to post transactions which don't fit into any of the child categories.

yagebu commented 7 years ago

@ngoonee: The budget on Expenses:Food is not necessary anymore. Since #521 budgets are accumulated where it makes sense. So now you can collapse any account to see the accumulated (including all children accounts) budget and balance. I've also just pushed a commit that makes the bar charts use this accumulated budget too.

ngoonee commented 7 years ago

@yagebu oh, that's exactly it! Thanks much.

beancount / fava

Budget Accumulation Issues #507