cant find file after watchtower update

[markwoodward86]

Good day!

I just updated both fireflyiii-core and the plaid connector with a watchtower run and now the plaid connector will not start up. I get exit code 0 with the error

Caused by: java.io.FileNotFoundException: /opt/fpc-cursors/plaid_sync_cursors.txt (Permission denied)

I have not changed anything else and have tried starting a new container from the latest image. I confirm that the filepath indeed does not exist (but it did not before either). Is there something different that i need to do to initiate this new image? The user does have write permissions to the opt directory.

Reverted to v1.0.8 and i am good again 👍

[dvankley]

What is the directory you're using to store the persistence file on your host machine? How have you set that as the value for HOST_PERSISTENCE_DIRECTORY_LOCATION in the environment docker compose uses? What are the permissions for that directory on your host machine?

This is the same symptom as https://github.com/dvankley/firefly-plaid-connector-2/issues/50 and your testing strongly points to it being an issue with the latest image. I haven't changed anything that should affect this stuff in the latest version, but a dependency update definitely could have.

[markwoodward86]

Thanks for the quick response. Permissions to my directory are full read write for the user. I performed a CHOWN o+x there again for good measure when testing yesterday before i reverted. Here is my compose file:

version: '3.8'

services:
  app:
    image: ghcr.io/dvankley/firefly-plaid-connector-2:v1.0.8
    restart: always
    volumes:
        - type: bind
          source: ~/plaid-connector-fireflyiii/firefly-plaid-connector-2/src/main/resources/application.yml
          target: /opt/fpc-config/application.yml
          read_only: true
        - type: bind
          source: ~/plaid-connector-fireflyiii/persistence
          target: /opt/fpc-cursors
    environment:
      - SPRING_CONFIG_LOCATION=/opt/fpc-config/application.yml
      - FIREFLYPLAIDCONNECTOR2_POLLED_CURSORFILEDIRECTORYPATH=/opt/fpc-cursors

volumes:
  fpc-cursors:

[dvankley]

Unfortunately I haven't been able to reproduce this locally; seems to work fine for me so far.

Permissions to my directory are full read write for the user.

For which user? The public/others user category? Can you post the ls -l permissions for your ~/plaid-connector-fireflyiii/persistence directory?

I performed a CHOWN o+x there

Do you mean you ran chmod o+x?

Maybe also try putting in the full absolute directory path instead of using ~?

[markwoodward86]

Unfortunately I haven't been able to reproduce this locally; seems to work fine for me so far.

Permissions to my directory are full read write for the user.

For which user? The public/others user category? Can you post the ls -l permissions for your ~/plaid-connector-fireflyiii/persistence directory?

Root user. ls -l output as follows

total 4 -rw-r--r-- 1 1000 1000 1005 Jan 2 16:55 plaid_sync_cursors.txt

I performed a CHOWN o+x there

Do you mean you ran chmod o+x?

sorry, yes

Maybe also try putting in the full absolute directory path instead of using ~?

I have tried this and updated to latest branch and got the same error as previously identified. I have reverted back to v1.0.8 again.

Happy to try other things though. I am not extremely well versed in linux but happy to try things. I am running this in a container on Proxmox 8.1.3 using docker-compose FWIW.

[dvankley]

What OS are you running? Still working on reproducing this, as shotgunning ideas doesn't seem to be getting the job done.

[markwoodward86]

Interesting. Im running this in an unprivileged Debian container on Proxmox 8.1.3 using docker-compose.

[dvankley]

@markwoodward86 sorry I haven't had a chance to dig much into this. My initial attempts to reproduce have been unsuccessful, but I have a couple more ideas to try soonish. I still suspect it might be a dependency issue, so if you have a chance to try the latest release https://github.com/dvankley/firefly-plaid-connector-2/releases/tag/v1.1.0 and see if that makes a difference, I'd appreciate it.

[markwoodward86]

Morning @dvankley here are the logs from 1.1.0 attempt:

Setting Active Processor Count to 2,
Calculating JVM memory based on 5062148K available memory,
For more information on this calculation, see https://paketo.io/docs/reference/java-reference/#memory-calculator,
Calculated JVM Memory Configuration: -XX:MaxDirectMemorySize=10M -Xmx4667507K -XX:MaxMetaspaceSize=87440K -XX:ReservedCodeCacheSize=240M -Xss1M (Total Memory: 5062148K, Thread Count: 50, Loaded Class Count: 13024, Headroom: 0%),
Enabling Java Native Memory Tracking,
Adding 137 container CA certificates to JVM truststore,
Spring Cloud Bindings Enabled,
Picked up JAVA_TOOL_OPTIONS: -Djava.security.properties=/layers/paketo-buildpacks_bellsoft-liberica/java-security-properties/java-security.properties -XX:+ExitOnOutOfMemoryError -XX:ActiveProcessorCount=2 -XX:MaxDirectMemorySize=10M -Xmx4667507K -XX:MaxMetaspaceSize=87440K -XX:ReservedCodeCacheSize=240M -Xss1M -XX:+UnlockDiagnosticVMOptions -XX:NativeMemoryTracking=summary -XX:+PrintNMTStatistics -Dorg.springframework.cloud.bindings.boot.enable=true,
,
  .   ____          _            __ _ _,
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \,
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \,
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) ),
  '  |____| .__|_| |_|_| |_\__, | / / / /,
 =========|_|==============|___/=/_/_/_/,
 :: Spring Boot ::                (v3.2.2),
,
2024-02-06T12:55:08.014Z  INFO 1 --- [           main] .d.f.FireflyPlaidConnector2ApplicationKt : Starting FireflyPlaidConnector2ApplicationKt v1.1.0 using Java 17.0.9 with PID 1 (/workspace/BOOT-INF/classes started by cnb in /workspace),
2024-02-06T12:55:08.017Z DEBUG 1 --- [           main] .d.f.FireflyPlaidConnector2ApplicationKt : Running with Spring Boot v3.2.2, Spring v6.1.3,
2024-02-06T12:55:08.019Z  INFO 1 --- [           main] .d.f.FireflyPlaidConnector2ApplicationKt : No active profile set, falling back to 1 default profile: "default",
2024-02-06T12:55:10.132Z  INFO 1 --- [           main] .d.f.FireflyPlaidConnector2ApplicationKt : Started FireflyPlaidConnector2ApplicationKt in 2.557 seconds (process running for 3.035),
2024-02-06T12:55:10.738Z DEBUG 1 --- [           main] n.d.f.sync.PolledSyncRunner              : Beginning Plaid sync endpoint cursor initialization,
2024-02-06T12:55:10.739Z DEBUG 1 --- [           main] n.d.f.sync.PolledSyncRunner              : Cursor map contains access-development-e41229b7-c791-4de7-84e5-c88f8d4e15c5, skipping initialization for it,
2024-02-06T12:55:10.739Z DEBUG 1 --- [           main] n.d.f.sync.PolledSyncRunner              : Cursor map contains access-development-3a465514-c01f-4b84-9bc4-88846bc40225, skipping initialization for it,
2024-02-06T12:55:10.739Z DEBUG 1 --- [           main] n.d.f.sync.PolledSyncRunner              : Cursor map contains access-development-df013456-3540-4ea4-9bf1-3a6eceae9def, skipping initialization for it,
2024-02-06T12:55:10.770Z ERROR 1 --- [           main] o.s.boot.SpringApplication               : Application run failed,
,
java.lang.reflect.UndeclaredThrowableException: Failed to invoke event listener method,
HandlerMethod details: ,
Bean [net.djvk.fireflyPlaidConnector2.FireflyPlaidConnector2Application$$SpringCGLIB$$0],
Method [public void net.djvk.fireflyPlaidConnector2.FireflyPlaidConnector2Application.appReady()],
Resolved arguments: ,
,
    at org.springframework.context.event.ApplicationListenerMethodAdapter.doInvoke(ApplicationListenerMethodAdapter.java:382) ~[spring-context-6.1.3.jar:6.1.3],
    at org.springframework.context.event.ApplicationListenerMethodAdapter.processEvent(ApplicationListenerMethodAdapter.java:237) ~[spring-context-6.1.3.jar:6.1.3],
    at org.springframework.context.event.ApplicationListenerMethodAdapter.onApplicationEvent(ApplicationListenerMethodAdapter.java:168) ~[spring-context-6.1.3.jar:6.1.3],
    at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:178) ~[spring-context-6.1.3.jar:6.1.3],
    at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:171) ~[spring-context-6.1.3.jar:6.1.3],
    at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:149) ~[spring-context-6.1.3.jar:6.1.3],
    at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:451) ~[spring-context-6.1.3.jar:6.1.3],
    at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:384) ~[spring-context-6.1.3.jar:6.1.3],
    at org.springframework.boot.context.event.EventPublishingRunListener.ready(EventPublishingRunListener.java:109) ~[spring-boot-3.2.2.jar:3.2.2],
    at org.springframework.boot.SpringApplicationRunListeners.lambda$ready$6(SpringApplicationRunListeners.java:80) ~[spring-boot-3.2.2.jar:3.2.2],
    at java.base/java.lang.Iterable.forEach(Unknown Source) ~[na:na],
    at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:118) ~[spring-boot-3.2.2.jar:3.2.2],
    at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:112) ~[spring-boot-3.2.2.jar:3.2.2],
    at org.springframework.boot.SpringApplicationRunListeners.ready(SpringApplicationRunListeners.java:80) ~[spring-boot-3.2.2.jar:3.2.2],
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:348) ~[spring-boot-3.2.2.jar:3.2.2],
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1354) ~[spring-boot-3.2.2.jar:3.2.2],
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1343) ~[spring-boot-3.2.2.jar:3.2.2],
    at net.djvk.fireflyPlaidConnector2.FireflyPlaidConnector2ApplicationKt.main(FireflyPlaidConnector2Application.kt:31) ~[classes/:1.1.0],
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na],
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:na],
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:na],
    at java.base/java.lang.reflect.Method.invoke(Unknown Source) ~[na:na],
    at org.springframework.boot.loader.launch.Launcher.launch(Launcher.java:91) ~[workspace/:na],
    at org.springframework.boot.loader.launch.Launcher.launch(Launcher.java:53) ~[workspace/:na],
    at org.springframework.boot.loader.launch.JarLauncher.main(JarLauncher.java:58) ~[workspace/:na],
Caused by: java.io.FileNotFoundException: /opt/fpc-cursors/plaid_sync_cursors.txt (Permission denied),
    at java.base/java.io.FileOutputStream.open0(Native Method) ~[na:na],
    at java.base/java.io.FileOutputStream.open(Unknown Source) ~[na:na],
    at java.base/java.io.FileOutputStream.<init>(Unknown Source) ~[na:na],
    at java.base/java.io.FileOutputStream.<init>(Unknown Source) ~[na:na],
    at kotlin.io.FilesKt__FileReadWriteKt.writeBytes(FileReadWrite.kt:108) ~[kotlin-stdlib-1.9.22.jar:1.9.22-release-704],
    at kotlin.io.FilesKt__FileReadWriteKt.writeText(FileReadWrite.kt:134) ~[kotlin-stdlib-1.9.22.jar:1.9.22-release-704],
    at kotlin.io.FilesKt__FileReadWriteKt.writeText$default(FileReadWrite.kt:134) ~[kotlin-stdlib-1.9.22.jar:1.9.22-release-704],
    at net.djvk.fireflyPlaidConnector2.sync.PolledSyncRunner$writeCursorMap$2.invokeSuspend(PolledSyncRunner.kt:286) ~[classes/:1.1.0],
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) ~[kotlin-stdlib-1.9.22.jar:1.9.22-release-704],
    at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108) ~[kotlinx-coroutines-core-jvm-1.7.3.jar:na],
    at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:115) ~[kotlinx-coroutines-core-jvm-1.7.3.jar:na],
    at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:103) ~[kotlinx-coroutines-core-jvm-1.7.3.jar:na],
    at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584) ~[kotlinx-coroutines-core-jvm-1.7.3.jar:na],
    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793) ~[kotlinx-coroutines-core-jvm-1.7.3.jar:na],
    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697) ~[kotlinx-coroutines-core-jvm-1.7.3.jar:na],
    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684) ~[kotlinx-coroutines-core-jvm-1.7.3.jar:na],
,
2024-02-06T12:55:10.781Z  INFO 1 --- [           main] n.d.f.sync.PolledSyncRunner              : Shutting down class net.djvk.fireflyPlaidConnector2.sync.PolledSyncRunner,
,
Native Memory Tracking:,
,
Total: reserved=5487961373, committed=290866461,
       malloc: 51836189 #239285,
       mmap:   reserved=5436125184, committed=239030272,
,
-                 Java Heap (reserved=4781506560, committed=138412032),
                            (mmap: reserved=4781506560, committed=138412032) ,
 ,
-                     Class (reserved=84626308, committed=5262212),
                            (classes #8190),
                            (  instance classes #7667, array classes #523),
                            (malloc=740228 #16161) ,
                            (mmap: reserved=83886080, committed=4521984) ,
                            (  Metadata:   ),
                            (    reserved=67108864, committed=31850496),
                            (    used=31626072),
                            (    waste=224424 =0.70%),
                            (  Class space:),
                            (    reserved=83886080, committed=4521984),
                            (    used=4274192),
                            (    waste=247792 =5.48%),
 ,
-                    Thread (reserved=22091536, committed=1742608),
                            (thread #22),
                            (stack: reserved=22020096, committed=1671168),
                            (malloc=42208 #160) ,
                            (arena=29232 #50),
 ,
-                      Code (reserved=254386096, committed=13569968),
                            (malloc=753584 #5076) ,
                            (mmap: reserved=253632512, committed=12816384) ,
 ,
-                        GC (reserved=216471819, committed=44169483),
                            (malloc=5323019 #3749) ,
                            (mmap: reserved=211148800, committed=38846464) ,
 ,
-                  Compiler (reserved=29743288, committed=29743288),
                            (malloc=47392 #394) ,
                            (arena=29695896 #13),
 ,
-                  Internal (reserved=309388, committed=309388),
                            (malloc=272524 #5825) ,
                            (mmap: reserved=36864, committed=36864) ,
 ,
-                     Other (reserved=24576, committed=24576),
                            (malloc=24576 #4) ,
 ,
-                    Symbol (reserved=10031768, committed=10031768),
                            (malloc=9663128 #205541) ,
                            (arena=368640 #1),
 ,
-    Native Memory Tracking (reserved=3835200, committed=3835200),
                            (malloc=6624 #96) ,
                            (tracking overhead=3828576),
 ,
-        Shared class space (reserved=16777216, committed=10866688),
                            (mmap: reserved=16777216, committed=10866688) ,
 ,
-               Arena Chunk (reserved=768296, committed=768296),
                            (malloc=768296) ,
 ,
-                   Tracing (reserved=345, committed=345),
                            (malloc=345 #9) ,
 ,
-                 Arguments (reserved=169, committed=169),
                            (malloc=169 #5) ,
 ,
-                    Module (reserved=64816, committed=64816),
                            (malloc=64816 #734) ,
 ,
-                 Safepoint (reserved=8192, committed=8192),
                            (mmap: reserved=8192, committed=8192) ,
 ,
-           Synchronization (reserved=59016, committed=59016),
                            (malloc=59016 #696) ,
 ,
-            Serviceability (reserved=672, committed=672),
                            (malloc=672 #6) ,
 ,
-                 Metaspace (reserved=67254160, committed=31995792),
                            (malloc=145296 #69) ,
                            (mmap: reserved=67108864, committed=31850496) ,
 ,
-      String Deduplication (reserved=704, committed=704),
                            (malloc=704 #8) ,
 ,
-           Object Monitors (reserved=1248, committed=1248),
                            (malloc=1248 #6) ,

[dvankley]

Same thing. Ah well, I guess that would have been too easy. Thanks for trying.

[markwoodward86]

@dvankley i did some chat gpt digging and i was able to get the 1.1.0 version working by running a chmod 777 on the persistence file. i originally tried simply granting read write access and had changed the user and group to root but this was not enough. after the 777 change the latest file version works.