search

dvankley / firefly-plaid-connector-2

Connector to pull Plaid financial data into the Firefly finance tool
GNU General Public License v3.0
94 stars 14 forks source link

java.lang.IllegalArgumentException: Can't get Plaid amount for a Firefly transaction of type opening balance #50

Closed hjmuller closed 7 months ago

hjmuller commented 8 months ago

I'm receiving the following error when trying to run polled transactions. It seems to have only started in the past few days.

java.lang.IllegalArgumentException: Can't get Plaid amount for a Firefly transaction of type opening balance

dvankley commented 8 months ago

Looks like a logic error that should be easily fixable. I'll cut a new release for this shortly. Not sure why it would only have happened recently. Did you create any new Firefly accounts that would have had opening balance transactions recently?

dvankley commented 8 months ago

@hjmuller please try https://github.com/dvankley/firefly-plaid-connector-2/releases/tag/v1.0.9

hjmuller commented 8 months ago

Now I get this error:

java.lang.reflect.UndeclaredThrowableException: Failed to invoke event listener method

dvankley commented 8 months ago

@hjmuller that's not enough information to diagnose the issue. Please post the whole stack trace.

hjmuller commented 8 months ago

My apologies. I had an issue with the syntax so I didn't share the full trace previously.

Please see below for the output.


root@prd-vm-docker:~/docker/firefly_iii_plaid_connector$ docker-compose --file docker-compose-polled.yaml up
Creating network "firefly_iii_plaid_connector_default" with the default driver
Creating firefly_iii_plaid_connector_app_1 ... done
Attaching to firefly_iii_plaid_connector_app_1
app_1  | Setting Active Processor Count to 6
app_1  | Calculating JVM memory based on 7809596K available memory
app_1  | For more information on this calculation, see https://paketo.io/docs/reference/java-reference/#memory-calculator
app_1  | Calculated JVM Memory Configuration: -XX:MaxDirectMemorySize=10M -Xmx7415068K -XX:MaxMetaspaceSize=87327K -XX:ReservedCodeCacheSize=240M -Xss1M (Total Memory: 7809596K, Thread Count: 50, Loaded Class Count: 13004, Headroom: 0%)
app_1  | Enabling Java Native Memory Tracking
app_1  | Adding 137 container CA certificates to JVM truststore
app_1  | Spring Cloud Bindings Enabled
app_1  | Picked up JAVA_TOOL_OPTIONS: -Djava.security.properties=/layers/paketo-buildpacks_bellsoft-liberica/java-security-properties/java-security.properties -XX:+ExitOnOutOfMemoryError -XX:ActiveProcessorCount=6 -XX:MaxDirectMemorySize=10M -Xmx7415068K -XX:MaxMetaspaceSize=87327K -XX:ReservedCodeCacheSize=240M -Xss1M -XX:+UnlockDiagnosticVMOptions -XX:NativeMemoryTracking=summary -XX:+PrintNMTStatistics -Dorg.springframework.cloud.bindings.boot.enable=true
app_1  | 
app_1  |   .   ____          _            __ _ _
app_1  |  /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
app_1  | ( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
app_1  |  \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
app_1  |   '  |____| .__|_| |_|_| |_\__, | / / / /
app_1  |  =========|_|==============|___/=/_/_/_/
app_1  |  :: Spring Boot ::                (v3.2.0)
app_1  | 
app_1  | 2024-01-01T18:13:13.779Z  INFO 1 --- [           main] .d.f.FireflyPlaidConnector2ApplicationKt : Starting FireflyPlaidConnector2ApplicationKt v1.0.9 using Java 17.0.9 with PID 1 (/workspace/BOOT-INF/classes started by cnb in /workspace)
app_1  | 2024-01-01T18:13:13.781Z DEBUG 1 --- [           main] .d.f.FireflyPlaidConnector2ApplicationKt : Running with Spring Boot v3.2.0, Spring v6.1.1
app_1  | 2024-01-01T18:13:13.782Z  INFO 1 --- [           main] .d.f.FireflyPlaidConnector2ApplicationKt : No active profile set, falling back to 1 default profile: "default"
app_1  | 2024-01-01T18:13:16.880Z  INFO 1 --- [           main] .d.f.FireflyPlaidConnector2ApplicationKt : Started FireflyPlaidConnector2ApplicationKt in 3.897 seconds (process running for 4.945)
app_1  | 2024-01-01T18:13:16.894Z DEBUG 1 --- [           main] n.d.f.sync.PolledSyncRunner              : Beginning Plaid sync endpoint cursor initialization
app_1  | 2024-01-01T18:13:16.895Z DEBUG 1 --- [           main] n.d.f.sync.PolledSyncRunner              : Cursor map contains access-development-[REDACTED], skipping initialization for it
app_1  | 2024-01-01T18:13:16.896Z DEBUG 1 --- [           main] n.d.f.sync.PolledSyncRunner              : Cursor map contains access-development-[REDACTED], skipping initialization for it
app_1  | 2024-01-01T18:13:16.896Z DEBUG 1 --- [           main] n.d.f.sync.PolledSyncRunner              : Cursor map contains access-development-[REDACTED], skipping initialization for it
app_1  | 2024-01-01T18:13:18.560Z DEBUG 1 --- [           main] n.d.f.sync.PolledSyncRunner              : Received initial batch of sync updates for access token access-development-[REDACTED]. Updating cursor map to next cursor: 
app_1  | 2024-01-01T18:13:18.560Z DEBUG 1 --- [           main] n.d.f.sync.PolledSyncRunner              : Cursor map contains access-development-[REDACTED], skipping initialization for it
app_1  | 2024-01-01T18:13:18.580Z ERROR 1 --- [           main] o.s.boot.SpringApplication               : Application run failed
app_1  | 
app_1  | java.lang.reflect.UndeclaredThrowableException: Failed to invoke event listener method
app_1  | HandlerMethod details: 
app_1  | Bean [net.djvk.fireflyPlaidConnector2.FireflyPlaidConnector2Application$$SpringCGLIB$$0]
app_1  | Method [public void net.djvk.fireflyPlaidConnector2.FireflyPlaidConnector2Application.appReady()]
app_1  | Resolved arguments: 
app_1  | 
app_1  |        at org.springframework.context.event.ApplicationListenerMethodAdapter.doInvoke(ApplicationListenerMethodAdapter.java:382) ~[spring-context-6.1.1.jar:6.1.1]
app_1  |        at org.springframework.context.event.ApplicationListenerMethodAdapter.processEvent(ApplicationListenerMethodAdapter.java:237) ~[spring-context-6.1.1.jar:6.1.1]
app_1  |        at org.springframework.context.event.ApplicationListenerMethodAdapter.onApplicationEvent(ApplicationListenerMethodAdapter.java:168) ~[spring-context-6.1.1.jar:6.1.1]
app_1  |        at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:178) ~[spring-context-6.1.1.jar:6.1.1]
app_1  |        at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:171) ~[spring-context-6.1.1.jar:6.1.1]
app_1  |        at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:149) ~[spring-context-6.1.1.jar:6.1.1]
app_1  |        at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:445) ~[spring-context-6.1.1.jar:6.1.1]
app_1  |        at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:378) ~[spring-context-6.1.1.jar:6.1.1]
app_1  |        at org.springframework.boot.context.event.EventPublishingRunListener.ready(EventPublishingRunListener.java:109) ~[spring-boot-3.2.0.jar:3.2.0]
app_1  |        at org.springframework.boot.SpringApplicationRunListeners.lambda$ready$6(SpringApplicationRunListeners.java:80) ~[spring-boot-3.2.0.jar:3.2.0]
app_1  |        at java.base/java.lang.Iterable.forEach(Unknown Source) ~[na:na]
app_1  |        at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:118) ~[spring-boot-3.2.0.jar:3.2.0]
app_1  |        at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:112) ~[spring-boot-3.2.0.jar:3.2.0]
app_1  |        at org.springframework.boot.SpringApplicationRunListeners.ready(SpringApplicationRunListeners.java:80) ~[spring-boot-3.2.0.jar:3.2.0]
app_1  |        at org.springframework.boot.SpringApplication.run(SpringApplication.java:341) ~[spring-boot-3.2.0.jar:3.2.0]
app_1  |        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1342) ~[spring-boot-3.2.0.jar:3.2.0]
app_1  |        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1331) ~[spring-boot-3.2.0.jar:3.2.0]
app_1  |        at net.djvk.fireflyPlaidConnector2.FireflyPlaidConnector2ApplicationKt.main(FireflyPlaidConnector2Application.kt:31) ~[classes/:1.0.9]
app_1  |        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
app_1  |        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:na]
app_1  |        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:na]
app_1  |        at java.base/java.lang.reflect.Method.invoke(Unknown Source) ~[na:na]
app_1  |        at org.springframework.boot.loader.launch.Launcher.launch(Launcher.java:91) ~[workspace/:na]
app_1  |        at org.springframework.boot.loader.launch.Launcher.launch(Launcher.java:53) ~[workspace/:na]
app_1  |        at org.springframework.boot.loader.launch.JarLauncher.main(JarLauncher.java:58) ~[workspace/:na]
app_1  | Caused by: java.io.FileNotFoundException: /opt/fpc-cursors/plaid_sync_cursors.txt (Permission denied)
app_1  |        at java.base/java.io.FileOutputStream.open0(Native Method) ~[na:na]
app_1  |        at java.base/java.io.FileOutputStream.open(Unknown Source) ~[na:na]
app_1  |        at java.base/java.io.FileOutputStream.<init>(Unknown Source) ~[na:na]
app_1  |        at java.base/java.io.FileOutputStream.<init>(Unknown Source) ~[na:na]
app_1  |        at kotlin.io.FilesKt__FileReadWriteKt.writeBytes(FileReadWrite.kt:108) ~[kotlin-stdlib-1.9.21.jar:1.9.21-release-633]
app_1  |        at kotlin.io.FilesKt__FileReadWriteKt.writeText(FileReadWrite.kt:134) ~[kotlin-stdlib-1.9.21.jar:1.9.21-release-633]
app_1  |        at kotlin.io.FilesKt__FileReadWriteKt.writeText$default(FileReadWrite.kt:134) ~[kotlin-stdlib-1.9.21.jar:1.9.21-release-633]
app_1  |        at net.djvk.fireflyPlaidConnector2.sync.PolledSyncRunner$writeCursorMap$2.invokeSuspend(PolledSyncRunner.kt:286) ~[classes/:1.0.9]
app_1  |        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) ~[kotlin-stdlib-1.9.21.jar:1.9.21-release-633]
app_1  |        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108) ~[kotlinx-coroutines-core-jvm-1.7.3.jar:na]
app_1  |        at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:115) ~[kotlinx-coroutines-core-jvm-1.7.3.jar:na]
app_1  |        at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:103) ~[kotlinx-coroutines-core-jvm-1.7.3.jar:na]
app_1  |        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584) ~[kotlinx-coroutines-core-jvm-1.7.3.jar:na]
app_1  |        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793) ~[kotlinx-coroutines-core-jvm-1.7.3.jar:na]
app_1  |        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697) ~[kotlinx-coroutines-core-jvm-1.7.3.jar:na]
app_1  |        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684) ~[kotlinx-coroutines-core-jvm-1.7.3.jar:na]
app_1  | 
app_1  | 2024-01-01T18:13:18.587Z  INFO 1 --- [           main] n.d.f.sync.PolledSyncRunner              : Shutting down class net.djvk.fireflyPlaidConnector2.sync.PolledSyncRunner
app_1  | 
app_1  | Native Memory Tracking:
app_1  | 
app_1  | Total: reserved=8440263695, committed=315782159
app_1  |        malloc: 76608527 #273783
app_1  |        mmap:   reserved=8363655168, committed=239173632
app_1  | 
app_1  | -                 Java Heap (reserved=7595884544, committed=125829120)
app_1  |                             (mmap: reserved=7595884544, committed=125829120) 
app_1  |  
app_1  | -                     Class (reserved=84732956, committed=6024220)
app_1  |                             (classes #9410)
app_1  |                             (  instance classes #8803, array classes #607)
app_1  |                             (malloc=846876 #18833) 
app_1  |                             (mmap: reserved=83886080, committed=5177344) 
app_1  |                             (  Metadata:   )
app_1  |                             (    reserved=67108864, committed=37683200)
app_1  |                             (    used=37487352)
app_1  |                             (    waste=195848 =0.52%)
app_1  |                             (  Class space:)
app_1  |                             (    reserved=83886080, committed=5177344)
app_1  |                             (    used=5016488)
app_1  |                             (    waste=160856 =3.11%)
app_1  |  
app_1  | -                    Thread (reserved=30509416, committed=2144616)
app_1  |                             (thread #29)
app_1  |                             (stack: reserved=30408704, committed=2043904)
app_1  |                             (malloc=58064 #232) 
app_1  |                             (arena=42648 #73)
app_1  |  
app_1  | -                      Code (reserved=254966456, committed=20159160)
app_1  |                             (malloc=1333944 #6857) 
app_1  |                             (mmap: reserved=253632512, committed=18825216) 
app_1  |  
app_1  | -                        GC (reserved=330218158, committed=53009070)
app_1  |                             (malloc=14305966 #6452) 
app_1  |                             (mmap: reserved=315912192, committed=38703104) 
app_1  |  
app_1  | -                  Compiler (reserved=12088812, committed=12088812)
app_1  |                             (malloc=68836 #630) 
app_1  |                             (arena=12019976 #20)
app_1  |  
app_1  | -                  Internal (reserved=355479, committed=355479)
app_1  |                             (malloc=318615 #7419) 
app_1  |                             (mmap: reserved=36864, committed=36864) 
app_1  |  
app_1  | -                     Other (reserved=57344, committed=57344)
app_1  |                             (malloc=57344 #12) 
app_1  |  
app_1  | -                    Symbol (reserved=11147800, committed=11147800)
app_1  |                             (malloc=10484608 #230229) 
app_1  |                             (arena=663192 #1)
app_1  |  
app_1  | -    Native Memory Tracking (reserved=4388040, committed=4388040)
app_1  |                             (malloc=7496 #107) 
app_1  |                             (tracking overhead=4380544)
app_1  |  
app_1  | -        Shared class space (reserved=16777216, committed=10866688)
app_1  |                             (mmap: reserved=16777216, committed=10866688) 
app_1  |  
app_1  | -               Arena Chunk (reserved=31716528, committed=31716528)
app_1  |                             (malloc=31716528) 
app_1  |  
app_1  | -                   Tracing (reserved=345, committed=345)
app_1  |                             (malloc=345 #9) 
app_1  |  
app_1  | -                 Arguments (reserved=169, committed=169)
app_1  |                             (malloc=169 #5) 
app_1  |  
app_1  | -                    Module (reserved=80312, committed=80312)
app_1  |                             (malloc=80312 #783) 
app_1  |  
app_1  | -                 Safepoint (reserved=8192, committed=8192)
app_1  |                             (mmap: reserved=8192, committed=8192) 
app_1  |  
app_1  | -           Synchronization (reserved=61016, committed=61016)
app_1  |                             (malloc=61016 #718) 
app_1  |  
app_1  | -            Serviceability (reserved=672, committed=672)
app_1  |                             (malloc=672 #6) 
app_1  |  
app_1  | -                 Metaspace (reserved=67267040, committed=37841376)
app_1  |                             (malloc=158176 #82) 
app_1  |                             (mmap: reserved=67108864, committed=37683200) 
app_1  |  
app_1  | -      String Deduplication (reserved=704, committed=704)
app_1  |                             (malloc=704 #8) 
app_1  |  
app_1  | -           Object Monitors (reserved=2496, committed=2496)
app_1  |                             (malloc=2496 #12) 
app_1  |

Docker compose file if needed:

version: '3.7'

services:
  app:
    image: ghcr.io/dvankley/firefly-plaid-connector-2:latest
    env_file: .env
    restart: unless-stopped
    volumes:
        - type: bind
          source: "${HOST_APPLICATION_CONFIG_FILE_LOCATION}"
          target: /opt/fpc-config/application.yml
          read_only: true
        - type: bind
          source: "${HOST_PERSISTENCE_DIRECTORY_LOCATION}"
          target: /opt/fpc-cursors
    environment:
      - SPRING_CONFIG_LOCATION=/opt/fpc-config/application.yml
      - FIREFLYPLAIDCONNECTOR2_POLLED_CURSORFILEDIRECTORYPATH=/opt/fpc-cursors

volumes:
  fpc-cursors:
dvankley commented 8 months ago

@hjmuller what is the directory you're using to store the persistence file on your host machine? How have you set that as the value for HOST_PERSISTENCE_DIRECTORY_LOCATION in the environment docker compose uses? What are the permissions for that directory on your host machine?

hjmuller commented 8 months ago

My persistence directory is in the same directory as the docker compose file. I have a .env file as follows:

HOST_APPLICATION_CONFIG_FILE_LOCATION=/root/docker/firefly_iii_plaid_connector/application.yaml
HOST_PERSISTENCE_DIRECTORY_LOCATION=/root/docker/firefly_iii_plaid_connector/persistence

The persistence directory has the following permissions:

drwxrwxrwx 2 root root 4.0K Dec 10 20:59 persistence
dvankley commented 8 months ago

Hm, ok. That seems like it should be fine.

What OS are you running? I'm trying to reproduce this issue (and https://github.com/dvankley/firefly-plaid-connector-2/issues/58 which I think has the same root cause) but having trouble so far.

hjmuller commented 8 months ago

The host is a Debian VM running on a Proxmox hypervisor.

dvankley commented 7 months ago

@hjmuller Please try updating to the latest version of the connector and running chown -R 1000:1000 persistence. 1000 is the user id of the cnb user that the connector runs under in Docker. If that still doesn't work, please try restarting your VM.

https://github.com/dvankley/firefly-plaid-connector-2/issues/58 appears to be a very similar issue and was resolved recently, so I'm hoping we can get you sorted as well.

hjmuller commented 7 months ago

Thanks, this seems to have done the trick!