search

MoneyTools / MyMoney.Net

MyMoney is a rich client .NET ClickOnce application for managing your personal finances. It is written entirely in C# and is designed for programmers who want easy access to their data and who want to quickly and easily add their own features. Your data will not be locked up in some proprietary format, it is yours to do with as you like.
https://moneytools.github.io/MyMoney.Net/
70 stars 22 forks source link

Past and future recuring transaction patterns #112

Closed andreva7 closed 4 months ago

andreva7 commented 10 months ago

It would be interesting to have a UI that shows recuring transactions. For both purchases and transfers (possibly paychecks and rent), show what transactions should be happening soon and their projected amount.

If something has happened three times in a row, even if the last month is missing (because I don't download on a continual basis) show what should be happening in the comin weeks or months. That can help predict if I have enough funds in an account to cover it.

I can write up more details about what I thought of if you want.

clovett commented 10 months ago

Great idea, I already have the logic to find such things because this is also used for auto-categorization. Where do you want these to show up, in a separate "report" showing predicted future transactions, or as some sort of greyed-out new "predicted future" rows in the transaction register? We'd also need a way to cancel such a prediction when you know you have terminated the recuring bill and then remember not to bring it back again.

andreva7 commented 10 months ago

I think this would be a separate report so I can see the next set of bills that will show up, no matter which account they may occur in.

clovett commented 10 months ago

I pushed a quick and dirty attempt at a future bills report. It finds some things, but is not yet able to find recurring patterns inside a given payee. For example, I have recurring bills to State Farm insurance for a bunch of different cars, each car is a very different amount, and the amounts change quite a lot depending on driver ages and age of car and so on and so this one is harder to match and predict right now. I'll have to think more on the math for that one. But simple recurring payments to things like Netflix seems to work. It will show a prediction for each month out one year and then the total amount. It also picks up some bogus things that just happen to fit a recurring pattern, like my bill to sidekick coffee below, and the european union tax witholding - I probably should eliminate certain investment categories :-)

image

clovett commented 10 months ago

I improved it further so it can pick out some of the state farm recurring bills by taking the Category into account. The side effect of this is you will get no reports on transactions that have no Category. Still can't handle my fleet of cars yet, that's going to take some Kmeans clustering or something...

andreva7 commented 10 months ago

These are all in the same category but the payee name just changed. Should this predict?

image

andreva7 commented 10 months ago

This bill did not predict either

image

clovett commented 10 months ago

yeah, right now the payee has to be the same, so the "ATTBILL PAYMENT" would have been found but it would have predicted a payment in december, but december is now the past and so it assumed the payments stopped :-) If you change the last payee to match it should work. You can use the Alias feature to rename them all using the regular expression "ATT.PAYMENT" and rename them to "AT&T" and

The second case the payments have varied too much it has statistics.stdev(x) / statistics.mean(x) of 0.25, and the cuttoff is 0.2, but I have an idea on how to help with that which is instead do a linear regression so it can take inflation into account along a slope then look for the variance against that line. The linear regression on Telus Comm is y = 14.524x + 90.687 and the sum of the differences is 115, the sqrt of that is 10.7 and 10.7 / the mean is 0.06 so that should be a strong signal that these bills are related.

jpdup commented 10 months ago

Could we simplify by just offering the user a manual action on a transaction for flagging that transaction has being recurring?

From there we propose a detected account, Payee, Candace and amount

The use can override the proposed default

You can also cancel them later

clovett commented 10 months ago

Yes, manual intervention is always a possibility, but sounds like a hassle for the end user. I implemented the linear regression idea and it popped up a bunch more useful recurring bills in my data raising my annual to $30k from $20k, (yikes) with only a couple false positives on a pizza joint we use a lot and a 401k fund payment I make every month, so give it a whirl and see what you think. I also allowed it to skip some months at the end to take late payments into account (but not so many that it picks up past recuring bills that are no longer a thing) and so it should now also pick up your AT&T payments, but I still recommend you use the alias feature to normalize those payee names. Should I remove investment accounts from this or not?

clovett commented 10 months ago

Ok, I made a couple more improvements. It now detects and predicts cyclical patterns in bills that have a seasonal nature like electricity and water and I also added a new Category Type called "RecurringExpense" which you can set manually to force a given category to appear on the future payments report. Now my recurring burn rate is up to $37k, ouch!

image

andreva7 commented 10 months ago

Thanks for the work on this feature. I’ve been looking at it, including switching a number of things to the Recurring expense category. Here is a first round of feedback:

The January section has bills for February. And that pattern continues.

@.***

February has a collection of entries, some of other months, some for other years

@.***

In the table above, DAN Services, was converted to a recuring payment, but was not paid in 2022 or 2023. I think if two payments are missed, the series can be assumed to be discontinued.

Some things that would help me:

Thanks for the quick work adding this feature!

-Andre

From: Chris Lovett @.> Sent: Friday, January 5, 2024 2:08 AM To: MoneyTools/MyMoney.Net @.> Cc: andreva7 @.>; Author @.> Subject: Re: [MoneyTools/MyMoney.Net] Past and future recuring transaction patterns (Issue #112)

Ok, I made a couple more improvements. It now detects and predicts cyclical patterns in bills that have a seasonal nature like electricity and water and I also added a new Category Type called "RecurringExpense" which you can set manually to force a given category to appear on the future payments report.

image.png (view on web)https://github.com/MoneyTools/MyMoney.Net/assets/3336415/ea77b8e7-ea47-4666-b732-9ddfd8db0c26

— Reply to this email directly, view it on GitHubhttps://github.com/MoneyTools/MyMoney.Net/issues/112#issuecomment-1878422185, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AJBV66BTCZD2U5V72S65HHLYM7GIZAVCNFSM6AAAAABBKRO4UKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZYGQZDEMJYGU. You are receiving this because you authored the thread.Message ID: @.***>

clovett commented 10 months ago

Woops, yeah, I had a bug in the date column of the report - should be fixed now. I also do a better job of snapping monthly bills to a monthly cycle so you don't get 2 of them showing up under 1 month on the 1st and 31st for example. I also made the other fix you wanted so 2 missed payments means payment was canceled.

The hyperlink feature I need to think about some more since it is not just selecting by category you would need to see a new type of transaction view that is selecting by "payee and category" which we don't currently support.

andreva7 commented 10 months ago

Thanks.

I think It would be fine for the hot link to the category to bring you to the whole category. If I click on the “recuring bill” that would be the place to have the full filter and just show the past recurrences of that bill.

-Andre

From: Chris Lovett @.> Sent: January 10, 2024 2:51 AM To: MoneyTools/MyMoney.Net @.> Cc: andreva7 @.>; Author @.> Subject: Re: [MoneyTools/MyMoney.Net] Past and future recuring transaction patterns (Issue #112)

Woops, yeah, I had a bug in the date column of the report - should be fixed now. I also do a better job of snapping monthly bills to a monthly cycle so you don't get 2 of them showing up under 1 month on the 1st and 31st for example. I also made the other fix you wanted so 2 missed payments means payment was canceled.

The hyperlink feature I need to think about some more since it is not just selecting by category you would need to see a new type of transaction view that is selecting by "payee and category" which we don't currently support.

— Reply to this email directly, view it on GitHubhttps://github.com/MoneyTools/MyMoney.Net/issues/112#issuecomment-1884613442, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AJBV66BUJ3SFKQJPTO3ZWVDYNZXAHAVCNFSM6AAAAABBKRO4UKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBUGYYTGNBUGI. You are receiving this because you authored the thread.Message ID: @.***>

clovett commented 4 months ago

I added the hyperlinks, will be in version 2.1.0.20. I also fixed the back button which was not working on reports, so you'd click the link, go to transactions, and back would not take you to the report. This is now fixed also. Please give it a try if you like it I will publish a new version.