OAuth Integration (Chase, possibly others)

[assumebreach]

Do you have any insight on how to deal with legacy Chase integrations? See this:

Chase: https://dashboard.plaid.com/activity/status/institution/ins_3

Appears that Chase is moving to Oauth, but the 'account picker' doesn't explicitly allow selection of ins_56 (associated with their new authentication flow).

[mbafford]

Since the above page requires authentication, here's the content:

Institution Deprecation Since Dec 1, 2021 • Last updated on Dec 1, 2021 You're viewing status data for ins_3, our legacy Chase integration. This data is relevant for existing ins_3 items. To view status data for our new Chase OAuth integration, please see status information for ins_56. If you are enabled, all new Chase item adds will be routed through ins_56. If you have questions please contact Plaid Support.

I have two Chase items for the same account, both are ins_3.

I recently (in Nov-2021) used this to set up a second item for the same Chase account, but it looks like I missed the cutoff for the forced ins_56.

As an aside, my existing ins_3 "works", but is not maintaining a persistent authenticated session for me (as a way of forcing the migration?) - I have to log in every time - to the point I used Python/Playwright to automate launching this project, walking through the "update account" flow, and fixing my authentication session with Chase so the sync works.

I have no idea what needs to be changed to support the OAuth flow - maybe some changes to the API calls to Plaid? I'll keep it in mind to look into, but can't promise anything. PRs/further investigation details are definitely welcome!

I have a bit of dread that the OAuth flow may have another gate against the free dev accounts, kind of like what USBank has done. So there may be some other factors Chase imposes that prevent using a dev account on the OAuth flow.

For what it's worth, OFX on Chase works fine, and I supplement my Plaid script with OFX fetches. You just don't get pending transactions that way.

[mbafford]

Surprise surprise:

https://plaid.com/docs/link/oauth/#request-production-access-from-plaid

Request Production access from Plaid Production access is a prerequisite for implementing OAuth. Plaid will contact you once your account has been enabled for Production. In the US and Canada, OAuth requires Production access; you cannot connect to US OAuth institutions in the Development environment without having received both OAuth approval and Production approval. You can, however, test OAuth in the Sandbox environment, using Sandbox-only institutions, without needing Production approval.

As suspected, Plaid (and maybe the banks) are trying to squeeze out the dev accounts - not by explicitly changing their "free forever" pricing tier, but by forcing you to have a production account to do anything actually useful.

I'd be fine with this if I thought I could actually be granted a Production account, but I'm pretty positive they won't actually grant one to an individual (I haven't tried, I'm just pessimistic about it). Their data safety requirements (for accessing my own data) will probably be too onerous.

I'd love to hear otherwise if anyone knows, though - the pricing for the production accounts is manageable for the level I'd be doing.

Anyway, I won't be modifying this to support OAuth, since I have no valid use-case for it in the near term as a dev account user. If someone wants to modify it to support OAuth in the dev sandbox, then I'll gladly accept a PR, although I doubt there's a lot of valid uses of this tool for production-level accounts.

[4everlove]

Looks like Plaid no longer supports the legacy Chase endpoint. I wonder if there is other workflow you'd use, given that OAuth requires prod access to Plaid.

[mbafford]

@4everlove Thank you for confirming. My Chase integration now fails to allow me to re-link, so I am no longer able to download Chase transactions using the legacy endpoint. I was hoping it was a temporary issue, but based on your comment and the length of time it's been failing for me (about 2 weeks? probably since 1-Oct), I assume it's an intended break on Plaid/Chase's part.

I currently rely on OFX and PDF statements parsing for Chase transactions. It's a little cumbersome to initially make Chase work with OFX (if I remember correctly), but once you've gone through the steps to enable the OFX access, it works pretty well. Occasionally I have had OFX not report transactions for a week or so while Plaid would report them, but I haven't noticed that happening recently.

[mbafford]

@4everlove I take that back about OFX with Chase. Chase has apparently disabled OFX as well.

https://www.quicken.com/support/why-am-i-receiving-message-about-my-chase-accounts

So I guess now it's down to manually downloading, scraping (which Chase has been very technically hostile to in the past), or subscribing to a third-party that Chase has blessed. I don't think Chase has an API individuals can access.

[csku]

Hi, I just came across plaid-sync and wanted to share my experience getting Chase data through plaid for personal finances. I applied for production access. I filled out the questionnaire on Plaid's website. I pretty much shared whatever personal security practices I follow and stated that I will only ever connect to my accounts, and possibly in the future my spouse's, with consent. I was approved for production in a few days and had OAuth access enabled for Bank of America, Wells Fargo, Capital One, US Bank, USAA and Robinhood.

Access to Chase took another week or two because apparently Chase wanted to review stuff too, but eventually access was granted. I then had an issue with my accounts not being visible through the API, but I deleted permissions from Chase online banking and went through Link again, with the account_selection_enabled flag set to true. Since then, I've been using Plaid for my personal finances, including data from Chase, without any issues.

Edit: I found Plaid support to be very helpful through the process, even though I was transparent about my use case and the fact that after getting approved for production - and therefore unlocking OAuth - I intended to keep using OAuth in the free dev environment.

[henryrobbins]

Ah! @csku this is super helpful. I didn't understand why I was able to connect to Chase in the development environment after I was approved for production access. This makes more sense now. I'm glad to hear someone else has had success with this!

[csku]

Likewise, glad someone else has got this working @henryrobbins. However, Plaid is going to decommission their dev environment in June this year. Their email does say "If you have unused items in the Development environment, you’ll be provided with some credits to test in our Production environment in a limited capacity." Further, it states that Plaid will be "introducing limited, free testing in our Production environment."

It's an annoyance, but if the impact is truly limited to pointing to production instead of dev, that won't be too bad.

[henryrobbins]

I saw this when looking through the docs. Here it says:

To simplify the development process, Plaid will be replacing the Development environment with the ability to test with real data for free directly in Production. On June 20, 2024, the Plaid Development environment will be decommissioned, and all Development Items will be removed. You may continue to test on the Development environment until this time.

Hopefully, this means pointing to production instead of dev as you said.

P.S. If it's helpful to you, I added plaid-sync to PyPI (see my fork). I was working on a project, onm, to replace Mint. Initially, I was re-implementing a lot of the functionality here. Thankfully, I found this repo soon after I started. Now, I intend for onm to wrap around plaid-sync and provide budgeting capabilities.

[csku]

I don't actually use plaid-sync - I use a hacky solution based on plaid2text with beancount and fava. I discovered plaid-sync well after I started using this setup and just put in a comment because my experience felt relevant to this thread.

onm sounds very promising - keen to see where it goes

[henryrobbins]

I hadn't come across plaid2text yet -- thanks for sharing! Is it your impression that plaid-sync and plaid2text serve roughly the same purpose? It seems plaid-sync has more activity recently (although less forks overall). I had found beancount but missed fava in my initial research. I am starting to question the novelty of my vision for onm. My original impression of beancount and hledger was they seemed pretty bulky for personal budgeting purposes. Assuming that is your use case, can you speak to that?

cc @mbafford I see you contributed to plaid2text previously. Can you speak to what you were trying to achieve by creating plaid-sync?

[csku]

plaid2text outputs transactions in beancount / ledger format, I don't believe plaid-sync does so. The default db for plaid2text is mongo. There is SQLite support but my fork, which has added some functionality, hasn't been tested with SQLite because I'm happy with mongo.

I'm not entirely sure what you mean by bulky, but beancount + fava is great for my personal finance needs. Not budgeting (I don't find budgeting useful), but accounting and reporting. Over the years I've tried everything from GnuCash to Quicken to Mint, but this setup works best. I can set up accounts the way I want to, I can capture complex transactions, I can filter reports the way I want to - it just works. As an aside, I think the problem with personal finance software generally is that everyone has different edge cases and it's hard to solve for that. I know many people who said Mint would work great for them if only it had this one feature - but that one feature was different for everyone. Beancount is customizable enough that I can solve for my own edge cases. And maybe I'm being naive but I think it could also be simple enough for people who don't want to be as involved, provided they get comfortable with double entry bookkeeping.

FWIW, I'd looked at firefly-iii before discovering beancount. It does budgeting natively (unlike beancount which requires some creativity) while also doing double entry bookkeeping. However, it is less flexible in account types. It actually forces you to treat credit cards as asset accounts, with the asset amount equal to the credit limit, which by itself is backwards enough to be a dealbreaker for me - but it clearly works for others.

[mbafford]

@henryrobbins wrote:

I see you contributed to plaid2text previously. Can you speak to what you were trying to achieve by creating plaid-sync?

My best guess is I wanted something that worked well with my beancount-import workflow. I also am a big fan of sqlite for personal databases and have too much work-related bad memories of using mongo to want to keep it a part of my personal life, so that might have been a factor.

As for bulky, beancount attracts a certain kind of person. The process and the obsession seems to be a core part of it. I'm not entirely sure what you mean by bulk, but there's a lot of moving pieces in my particular setup and it's definitely not streamlined or convenient enough that I keep it updated as consistently as I'd like. But part of that just comes down to being the kind of person who seems to shift obsessions fairly frequently. I do inherently like beancount + beancount-import + fava. The fact I can entirely control every aspect of the data is nice - too many times I've run into limitations in some finance product and just given up on it. The main issues I have right now comes down to a) speed (beancount/fava can be quite slow on my journal) and b) lack of streamlined data ingest/input process (mainly my fault).

As for the actual purpose of this thread and the Plaid updates mentioned above (which weren't actually communicated to me by Plaid...) - I've tried to sign up for a production account on Plaid and on MX and both have ghosted me entirely. The form says "we'll get back to you in a few business days" and it's been months on both. I just gave up and went with https://beta-bridge.simplefin.org/ - which isn't really ideal for a lot of reasons, but it does work for my problematic Plaid accounts (which sounds like will soon be all of them).