search

wesselt / bunq2ynab

Upload bunq transactions to ynab
Other
78 stars 22 forks source link

Integrate with AWS Lambda - Requirements file, .gitignore updates, integrate SSM, SAM template. #22

Closed javydekoning closed 4 years ago

javydekoning commented 4 years ago

adding requirements file and updating .gitignore

wesselt commented 4 years ago

Looks good 👍 Added some comments! I'll try to set up a lambda at AWS to see it in action

javydekoning commented 4 years ago

Looks good 👍 Added some comments! I'll try to set up a lambda at AWS to see it in action

Cool, let me know if you need any help with that!

wesselt commented 4 years ago

Cool, let me know if you need any help with that!

Got to this point. It's probably Google-able, will try further this weekend!

2020-08-13 14:50:53,368 | CRITICAL | config.py:76 | read_ssm_config | Error loading configuration from SSM Parameter: bunq2ynab-config: An error occurred (AccessDeniedException) when calling the GetParameter operation: User: arn:aws:sts::122776901776:assumed-role/bunq2ynab-1aws-2aws-role-254ipodc/bunq2ynab-1aws-2aws is not authorized to perform: ssm:GetParameter on resource: arn:aws:ssm:eu-central-1:122776901776:parameter/bunq2ynab-config

javydekoning commented 4 years ago

Looks like you didn't deploy from the template. Your lambda is missing a role with permission to read from ssm. Let me send you instructions tomorrow :-)

wesselt commented 4 years ago

Yeah, I thought if I'd take the template from the serverless app store, I'd get the old version?

Looking forward to how to add the permissions.

javydekoning commented 4 years ago

Yeah, I thought if I'd take the template from the serverless app store, I'd get the old version?

Looking forward to how to add the permissions.

Yes, I didn't publish this version yet as I didn't have time to run tests. I'll reach out tomorrow morning.

javydekoning commented 4 years ago

To build and deploy you need SAM. (Users will not need this, they will be able to install from the repo).

Instructions to get it running:

sam build
sam deploy --guided

Stack Name [sam-app]: b2ytest
AWS Region [us-east-1]: eu-west-1
Parameter BunqApiToken []: <removed>
Parameter YnabAccessToken []: <removed>
Confirm changes before deploy [y/N]: y
#SAM needs permission to be able to create roles to connect to the resources in your template
Allow SAM CLI IAM role creation [Y/n]: y
Save arguments to samconfig.toml [Y/n]: n

It will immediately error out due to an argparse error, but I don't have time to debug it today. Maybe I can spend some time on it on Sunday. Shouldn't be to difficult.

wesselt commented 4 years ago

Just merged this request from the command line. Because I rebased the pull request doesn't seem to notice? Or now it does.

I'm still trying to get SAM to install. I've got it in Windows but then it doesn't work in WSL. Arch Linux doens't seem to have a package for it, and it doesn't work on my Pi.

I was able to manually add the rights to the parameters in IAM > Access management > Roles. Still have to figure out how this works with encrypted parameters.

Split SSM_PARAM into SSM_CONFIG_PARAM and SSM_STATE_PARAM. The latest version gets to a point where bunq says the API key is invalid. Curious how far you get!

wesselt commented 4 years ago

Got some more tries in, it now works for me! The "import boto3" is now in a separate function that is only called when SSM_CONFIG_PARAM or SSM_STATE_PARAM is set.

I changed it so it doesn't log the state or configuration and the keys and tokens in them.

javydekoning commented 4 years ago

That's great! Thanks. I'm still a little busy this week (holiday coming up :-) ). When I have some room I'll migrate my sync over to this version and check whether all will be ok.

wesselt commented 4 years ago

That'd be nice, checking is very useful! Next up for me is a look at API gateways. Maybe we could add that to the template at some point.

javydekoning commented 4 years ago

I checked and fixed a few things.

I'll now have it run for a couple of days to make sure everything works as expected. Adding a callback would be super easy.

wesselt commented 4 years ago

Thanks for testing! I can't see your code changes. Maybe because this pull request is already merged?

javydekoning commented 4 years ago

I comitted to config branch: https://github.com/wesselt/bunq2ynab/commit/08a464654074631e22b94feaa0869bbd329325a2

javydekoning commented 4 years ago

Update, running fine until now. No issues. If this continues to work I'll push this version to the Serverless Application Repository. I'll create issues for the remaining tasks.