theY4Kman / python-pocketsmith-web-client

Pocketsmith web/realtime client, for things the API does not provide
https://pypi.org/project/pocketsmith-web-client/
MIT License
1 stars 0 forks source link

Thoughts on usage #1

Open mbadran opened 3 years ago

mbadran commented 3 years ago

Hey there,

This is very cool, thanks for putting it out there. I'll take it for a spin soon.

You're 100% correct about the gaps in the PocketSmith API and the official web app. I've been hacking on the API (using your client!) to develop similar features, namely notifications for specific transactions (eg. salary payment). I've got that part working using the list_transaction_account_transactions_with_http_info resource, but your implementation is more elegant.

This has all got me thinking about taking an event-driven approach to personal finance, with rules that:

  1. Search for specific transactions or balances (eg. monthly salary payment or low account balance),
  2. Generate events when matching transactions are found,
  3. And then have those events trigger any number of configurable actions (eg. push notifications).

The formula looks like this:

Rule --> Event(s) --> Action(s)

A rule configuration could look like this:

rule = {
    "id": "b4d651ad",
    "name": "Salary",
    "description": "Monthly salary from my employer",
    "account_id": 716042,
    "transaction_type": "credit",
    "search_terms": "Salary",
    "amount": 500,
    "tags": ["salary", "employment"]
}

And the corresponding actions could look like this (on a high level):

(A stretch goal is to build this in such a way that PocketSmith is one of many potential sources, instead of having it tightly coupled.)

I'll take a shot at prototyping this as a backend API on Lambda/DynamoDB (using some of the code in this repo), and will share the results back. Would be keen to hear your thoughts, in the meanwhile.

theY4Kman commented 3 years ago

Well, the web client is definitely slower than the API, and though Pusher provides realtime info, it's usually limited to events telling the dashboard "hey, there's new data, you should reload". Also, I think I found the search a little limiting (though, I forget in which ways).

For my personal dashboard, I ended up syncing transactions to a local DB — I have an hourly cron job that syncs the last 30 days of data, and an every-12-hours job that does a full sync of the entire history. The rate limits on the API are really generous, and reset every hour, I believe.

With all the data in my local DB, I've found it a lot easier to play with. I'll see if I can't scrub that code a wee bit and throw it in a repo for your perusal — maybe it gives ya some ideas.

Regardless of how you do the transactions, there's one thing to be aware of: Yodlee (the finance data provider for Pocketsmith) treats pending transactions as separate from completed transactions, in the sense they have different transaction IDs. There doesn't seem to be any explicit way to link old pending transactions to their completed counterparts — the only way to know a transaction is no longer pending is to check whether it exists with the API (or when it no longer appears in the transactions list endpoint). That's actually why I do a full sync every 12 hours — just in the bizarre case some transaction is pending for more than 30 days.


All that said, having a reactive solution like this was a huge reason for building these Pocketsmith clients. I just wish my banks had APIs to make transfers =/

With a rule-based system, one inevitably wants more and more complex rules; it might be wise to hook into an existing "automation" system, like NodeRED or n8n — becoming a provider of triggers/events, and piggy-backing on their infinitely-configurable reactive rules systems, as well as their existing library of hooks (like push notifications, emails, SMS, etc).

mbadran commented 3 years ago

Sorry for the late reply. That's a fantastic overview, thanks! I wasn't aware of those automation systems, will definitely check them out.

The PocketSmith search API is limited because you can only search by keywords across all the transaction attributes. There's no way to specify an attribute and value in a search (eg. amount=500). It definitely makes sense to pull all the transactions into a local DB, as you've done, which would then allow for more expressive searching.

It's a bummer that Yodlee isn't smart enough to link pending and confirmed transactions. That could explain why PocketSmith is upgrading to the latest Yodlee APIs -- https://learn.pocketsmith.com/article/1024-what-were-working-on-now#rebuilding

Another limitation of working with the PocketSmith API is that you're bound to their feed syncing schedule. If you want to find out about a transaction within minutes of it occurring at the bank, you're out of luck. (This is also assuming that Yodlee is responsive and quickly publishes the bank data to their feeds.) All in all, PocketSmith is not built to be a real-time system, so reactive scenarios will be somewhat hampered by that dynamic.

And great point on the bank APIs! Things are moving in that direction with initiatives like Open Banking, so hopefully, fintech will be much more developer-friendly over the next few years.