Open idpaterson opened 9 months ago
Can you please provide a simple summary of why and where we need to be clicking this new button? It's easy to get lost in all the details in your OP.
Thanks for following up on this, it looks like Mint has pushed the sunset date so this might still have a chance to help people.
This adds a third button to the extension popup that should be visible when you click the extension icon. The button is enabled if you're viewing a Mint trend page and when clicked exports daily balances for that trend over the time period shown in Mint. Unlike the existing per-account export it allows combining accounts to export trends like "all student loan payoffs" or net worth.
When I was using this back in November to export very old trends I noticed that the Mint API took a lot longer to respond (and often timed out with no data at all) for old data. Severe rate limiting (like 1 request per 5 seconds) had no effect, it seemed to be a general Mint API issue. The Mint UI similarly failed to load trends most of the time.
If this is still an issue, a possible improvement would be to fetch data in the reverse order, collecting daily balances for the most recent transactions first moving back in time until the API gives up. A partial export could be more useful than no data at all.
Honestly though I don't know if anyone other than me finds this useful. I considered asking people on the subreddit but assumed that the Monarch dev team would not have time to review this one anyway.
Thank you for the quick follow-up and notes. I'll definitely what you've said about old trends in-mind when I export my Mint data near the end of December.
When I'm ready to export, I plan to combine the active PRs into my own fork and then use that instead of the current version of the plugin. I'm hoping (likely in vain) that Monarch addresses the open issues and PRs before then.
So there is at least one person who isn't you who finds what you've done useful. Thank you for taking the time to share your work.
Thanks! I assumed you were from Monarch 😄
I'll do some more testing on this to see if it can be improved and to see if the Mint API is still in a woeful state.
Today's update addresses issues I was having with pulling the current trend state from localStorage
. Mint tracks the trend there for the purposes of resuming it if you refresh the page, but it is extremely unreliable. Occasionally accounts get unset, dates change, and it is even more unpredictable if you have multiple tabs open. It often did not accurately reflect the UI state unless you had just recently configured the trend and you can see this in Mint by refreshing the trend page and occasionally ending up at a different trend.
Now after excessive digging around in the DOM to find the pieces that I needed, then discovering a Redux store with all of the data in one place, the extension instead pulls the current trend state from React. This works with multiple browser tabs open and avoids the debacle of transforming deselectedAccounts
into the selected accounts.
This does not yet support category, tag, and merchant filters available on the Income and Spending trends. I have started working on that because I have several tags that I like to track with annual trends. I can just get the data from Monarch now but hopefully it will be helpful to others.
Oh also the Mint API is speedy again, at least for me. No rate limiting issues so far testing this out.
Earlier I had interpreted a response with no Trend
array as a timeout, but after adding support for exporting trends filtered by tag or category I noticed that Trend
is always omitted when every value in the time period is zero. I was exporting some very infrequent tags that had large gaps between old and new data. The old data loaded fine but the middle gap always errored out. Since there are no dates in the response the extension just generates the rows with zero amounts.
This was great news since my past experience with API issues may have just been due to exporting gappy data.
Here are some examples of things you can export with the "Download current trend daily balances" button:
@idpaterson we're really sorry we're so behind on reviewing things here.. one thing that could help, could we split out any code in here related to rate-limiting of the existing functionality from the new functionality into separate PRs?
Good idea, at the time it was the only way I could make progress on this branch but it's a good idea to separate it. I'll check for any other standalone bug fixes, too.
Thanks for reviewing the small ones, I’ll rebase this PR when I get a chance!
@idpaterson Is this PR ready for review? Thanks!
Yep it is up to date with main and ready to review.
Any updates on this?
It’s not going to happen, Monarch definitely has more important work to do and we’re closing in on the Mint shutdown date.
It’s not going to happen
Seems you were right. :\ Thank you for trying to get this merged, regardless.
You’re welcome! Fortunately most of the improvements along the way to the trend button made it in via separate pull requests.
Monarch has been very nice to work with, they reached out recently offering to add more code review time for contributions to this extension. However, with the shutdown looming I decided it was not worth taking the risk of pushing new code. If an undetected mistake were introduced in the past couple weeks it could have left a lot of people with imperfect CSVs.
Thanks for following up, I hope your migration went well
This update ports over the "Export Daily History to CSV" button from my userscript now leveraging the API framework of Monarch's extension along with several other enhancements. Since inserting the export button into the page did not work out this pull request accomplishes the same with a "Download current trend daily balances" button in the popup. No content script so none of the insurmountable issues that derailed #23.
The button is enabled when viewing an exportable trend like Net Income Over Time or Spending Over Time for 5 selected accounts.
Export any time-based trend
The fetch functions have been generalized to work for any time-based trend which is pretty much just passing in a report type and a list of accounts that become filters. When working with trend state the extension fetches the daily balances for the exact time period you're looking at – no optimization to avoid fetching zero months. I had it in the original userscript but since the popup is such a nicer way to export all accounts I think the use cases for trend exporting would not demand that kind of optimization. Just use a custom date range if you're concerned about loading too much data 😉
The
formatBalancesAsCSV
function now supports the oddball net worth and net income trends that include extra columns.zipTrendEntries()
handles the API response for those trends since they can contain twoTrendEntry
rows per date. Same trimming of zero values at the end of the CSV for all trend types.Making API requests correspond to Mint trend state
The current trend state is sourced from a Redux store on a component in the Mint Trends page. It provides all of the data necessary to reproduce the same API queries that Mint used to chart the trend.
This Redux store was difficult to find and the original implementation used a different data source that was convoluted and unreliable. Original discussion is no longer relevant, but preserved below: