Setup Bunq to Ynab sync

mvgijssel commented 1 year ago

Bunq has no native integration into YNAB, so this requires an external library https://github.com/wesselt/bunq2ynab. Let's figure out how we can run the library using Bazel. A next step can be to package it up into a docker image (with apko) and run this for example in the provisioner.

[x] Create bunq2ynab docker image (https://github.com/vgijssel/setup/pull/544)
[x] Setup release flow docker image (https://github.com/vgijssel/setup/pull/601)
[x] Create multi-arch image for both linux amd64 and linux arm64 (https://github.com/vgijssel/setup/pull/614)

[x] Test if multi-arch image works inside provisioner

bazel run //:tsh -- login
bazel run //:tsh -- ssh ubuntu@provisioner
docker run --rm -it --entrypoint="" ghcr.io/vgijssel/setup/bunq2ynab:0.1.0 /bin/bash

Works 🎉

[x] Wire bunq2ynab into provisioner (https://github.com/vgijssel/setup/pull/616)
[x] Fix provisioner deployment (https://github.com/vgijssel/setup/pull/617, https://github.com/vgijssel/setup/pull/618, https://github.com/vgijssel/setup/pull/619, https://github.com/vgijssel/setup/pull/620, https://github.com/vgijssel/setup/pull/621)
[x] Reduce how often validate script runs to fix rate limit with 1password
[x] Reduce how often bunq2ynab restarts to reduce rate limit
[x] Fix command substitution not erroring out (https://github.com/vgijssel/setup/pull/623)
[x] Fix filtering out deleted git files (https://github.com/vgijssel/setup/pull/624)
[x] Fix infinite deployments (https://github.com/vgijssel/setup/pull/625)
[x] Do a manual deploy now that 1Password is once more rate limited
[x] Fix github action (https://github.com/vgijssel/setup/pull/626)
[x] Fix the actual deployment by loading files correctly (https://github.com/vgijssel/setup/pull/627)
[x] Fix provisioner deploy again (https://github.com/vgijssel/setup/pull/633)
[x] Validate if we need to define state file location
[ ] Setup monitoring

mvgijssel commented 1 year ago

Let’s try to create a container image with Apko by zipping up the Bazel target with https://github.com/bazelbuild/rules_pkg and unzipping it in the image!

mvgijssel commented 1 year ago

Asking Bazel community https://bazelbuild.slack.com/archives/CA31HN1T3/p1698678406629479

mvgijssel commented 10 months ago

Manual release

export GITHUB_TOKEN=$(bazel run //:op -- read "op://vgijssel-prod/github-release/credential")

bazel run //:release_manager.version

~/Development/setup master 6s
(devbox) nix-shell-env ❯ bazel run //:release_manager.version
INFO: Analyzed target //:release_manager.version (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //:release_manager.version up-to-date:
  /private/var/tmp/_bazel_maarten/8c4bfcd606c4d7b95cf418885251b08c/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/release_manager.version
INFO: Elapsed time: 0.314s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
INFO: Running command line: /private/var/tmp/_bazel_maarten/8c4bfcd606c4d7b95cf418885251b08c/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/release_manager.version
$ $'../rules_release~override/release/changesets_cli.sh' version
$ cd /Users/maarten/Development/setup
🦋  All files have been updated. Review them and commit at your leisure
Updated version for release provisioner from 1.0.1 to 1.0.2
Updated version for release bunq2ynab from 0.2.0 to 0.2.1
Updated version for release rules_release from 1.2.0 to 1.2.1
Updated version for release rules_task from 2.0.1 to 2.0.2

bazel run //:release_manager.publish

~/Development/setup master* 8s
(devbox) nix-shell-env ❯ bazel run //:release_manager.publish
INFO: Analyzed target //:release_manager.publish (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //:release_manager.publish up-to-date:
  /private/var/tmp/_bazel_maarten/8c4bfcd606c4d7b95cf418885251b08c/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/release_manager.publish
INFO: Elapsed time: 3.725s, Critical Path: 3.38s
INFO: 21 processes: 1 internal, 12 darwin-sandbox, 8 local.
INFO: Build completed successfully, 21 total actions
INFO: Running command line: /private/var/tmp/_bazel_maarten/8c4bfcd606c4d7b95cf418885251b08c/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/release_manager.publish
$ /private/var/tmp/_bazel_maarten/8c4bfcd606c4d7b95cf418885251b08c/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/release_manager.publish.runfiles/_main/provisioner/publish_github_release
https://github.com/vgijssel/setup/releases/tag/provisioner-v1.0.2
$ /private/var/tmp/_bazel_maarten/8c4bfcd606c4d7b95cf418885251b08c/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/release_manager.publish.runfiles/_main/tools/bunq2ynab/publish_oci_image
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
Login Succeeded
2023/12/20 19:34:40 existing blob: sha256:e75eb2190b1038bc4d0052198a47245af8adacd52b5eac6f4100b490fca59b5d
2023/12/20 19:34:40 existing blob: sha256:c71be7059a046f007116c0045d93657fb2a7e88a37b0e3e1dbd98005102c2cbd
2023/12/20 19:34:40 existing blob: sha256:49a0a9cdc58674b8b5e702119b3173dc84540e7c81ec33d9618f93243f9cfd50
2023/12/20 19:34:40 existing blob: sha256:ad259224458069def550d51a00302f36367757465f2fed5b70908a5de76cbac8
2023/12/20 19:34:41 pushed blob: sha256:98aed46db084a6dcd4a2c6f69720824eb4b6ec8b823771a2219499634c1e2a80
2023/12/20 19:34:41 pushed blob: sha256:772367c5259447875ac396021f3fd5c592769d02ae496c063a56c2e7d7d05979
2023/12/20 19:34:58 pushed blob: sha256:19a1e4d912506561fdd212892266b9d2c21beadaba32b4c3d6ffed12d97283d4
2023/12/20 19:34:58 pushed blob: sha256:6ae871e79ea56969808915ffefd161ddc1016117377b0568b430bba2ba698c08
2023/12/20 19:34:58 ghcr.io/vgijssel/setup/bunq2ynab@sha256:f0da104e7d4ec88990b004b14179aa44b94bfb50bbc184c4bb4186057c5586ac: digest: sha256:f0da104e7d4ec88990b004b14179aa44b94bfb50bbc184c4bb4186057c5586ac size: 757
2023/12/20 19:34:59 ghcr.io/vgijssel/setup/bunq2ynab@sha256:a27e9ae709d12663d500c9b4eaec317df76ad817a349c1d35855ee23f9831730: digest: sha256:a27e9ae709d12663d500c9b4eaec317df76ad817a349c1d35855ee23f9831730 size: 757
2023/12/20 19:34:59 ghcr.io/vgijssel/setup/bunq2ynab@sha256:93f32dbd239108c81ec58f7d52650c10db67a7a0cea4d8fcc06069debc469034: digest: sha256:93f32dbd239108c81ec58f7d52650c10db67a7a0cea4d8fcc06069debc469034 size: 669
2023/12/20 19:35:02 existing manifest: sha256:a27e9ae709d12663d500c9b4eaec317df76ad817a349c1d35855ee23f9831730
2023/12/20 19:35:02 existing manifest: sha256:f0da104e7d4ec88990b004b14179aa44b94bfb50bbc184c4bb4186057c5586ac
2023/12/20 19:35:03 ghcr.io/vgijssel/setup/bunq2ynab:0.2.1: digest: sha256:93f32dbd239108c81ec58f7d52650c10db67a7a0cea4d8fcc06069debc469034 size: 669
$ /private/var/tmp/_bazel_maarten/8c4bfcd606c4d7b95cf418885251b08c/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/release_manager.publish.runfiles/_main/tools/bunq2ynab/publish_github_release
https://github.com/vgijssel/setup/releases/tag/bunq2ynab-v0.2.1
$ $'/private/var/tmp/_bazel_maarten/8c4bfcd606c4d7b95cf418885251b08c/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/release_manager.publish.runfiles/rules_release~override/publish_github_release'
https://github.com/vgijssel/setup/releases/tag/rules_release-v1.2.1
$ $'/private/var/tmp/_bazel_maarten/8c4bfcd606c4d7b95cf418885251b08c/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/release_manager.publish.runfiles/rules_task~override/publish_github_release'
https://github.com/vgijssel/setup/releases/tag/rules_task-v2.0.2
$ /private/var/tmp/_bazel_maarten/8c4bfcd606c4d7b95cf418885251b08c/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/release_manager.publish.runfiles/_main/push_git_changes
Files changed:
.changeset/bunq2ynab-dull-radios-sniff.md .changeset/provisioner-tender-hounds-shout-2.md .changeset/provisioner-tender-hounds-shout.md .changeset/rules_release-heavy-laws-jam.md .changeset/rules_task-twenty-jeans-exist.md provisioner/CHANGELOG.md provisioner/version.txt rules/rules_release/CHANGELOG.md rules/rules_release/version.txt rules/rules_task/CHANGELOG.md rules/rules_task/version.txt tools/bunq2ynab/CHANGELOG.md tools/bunq2ynab/version.txt
Lefthook v1.4.8
RUNNING HOOK: pre-commit
black: (skip) no files for inspection
buildifier: (skip) no files for inspection

  EXECUTE > prettier
provisioner/CHANGELOG.md 23ms
rules/rules_release/CHANGELOG.md 11ms
rules/rules_task/CHANGELOG.md 4ms
tools/bunq2ynab/CHANGELOG.md 5ms

SUMMARY: (done in 1.47 seconds)
✔️  prettier
[master 34f5a89] chore(release_manager): update release files
 13 files changed, 40 insertions(+), 29 deletions(-)
 delete mode 100644 .changeset/bunq2ynab-dull-radios-sniff.md
 delete mode 100644 .changeset/provisioner-tender-hounds-shout-2.md
 delete mode 100644 .changeset/provisioner-tender-hounds-shout.md
 delete mode 100644 .changeset/rules_release-heavy-laws-jam.md
 delete mode 100644 .changeset/rules_task-twenty-jeans-exist.md
remote: Bypassed rule violations for refs/heads/master:
remote:
remote: - Changes must be made through a pull request.
remote:
remote: - 2 of 2 required status checks are expected.
remote:
remote:
remote: GitHub found 3 vulnerabilities on vgijssel/setup's default branch (1 high, 2 moderate). To find out more, visit:
remote:      https://github.com/vgijssel/setup/security/dependabot
remote:
To https://github.com/vgijssel/setup.git
   e0b73ea..34f5a89  HEAD -> master

mvgijssel commented 9 months ago

Works just fine without persisting the state file!

vgijssel / setup

Setup Bunq to Ynab sync #451