search

dvankley / firefly-plaid-connector-2

Connector to pull Plaid financial data into the Firefly finance tool
GNU General Public License v3.0
92 stars 14 forks source link

Connector crashes when trying to get foreign amount #71

Closed AmineI closed 5 months ago

AmineI commented 6 months ago

Hi, Connector crashes if there is a Firefly transaction with a foreign amount in firefly. I'll be sending a quick PR

dvankley commented 6 months ago

The change you reference here reverts a change I made recently to fix an issue caused by changes in Firefly.

What version of Firefly and the connector are you running?

AmineI commented 6 months ago

Ah, interesting. Apologies, I went super fast with this one.

On my end, I'm using auto updated docker setups for the connector and firefly.

That means Firefly 6.1.9 and 1.1.0 ( https://github.com/dvankley/firefly-plaid-connector-2/commit/d993d7c3ebdf5cf4820c23adc0dae65cfa23e6a7 ) for the connector.

Simple repro steps :

  1. create an account in Japanese Yen
  2. Create a transfer from your main account to the Yen account, with foreign amount filled out, like 30000, and a main amount that is not integer, such as 92.87
  3. Save the transfer
  4. Configure the connector to sync with the yen account, with a matching window big enough to pick up that transaction. Start the connector. Notice that the connector crashes after giving an "expected a int" error for foreign amount.

EDITED repro steps in bold

dvankley commented 6 months ago

Ok, thanks. That sounds pretty straightforward to reproduce. I'll take a look when I can.

dvankley commented 6 months ago

I tried to reproduce following your steps and didn't see any errors. Can you post an error log?

AmineI commented 6 months ago

Thanks for looking into it. Apologies for the delay - I couldn't get the correct log during the week. Here is my first log output :

"2024-02-26T16:42:14.216Z  INFO 1 --- [           main] .d.f.FireflyPlaidConnector2ApplicationKt : Starting FireflyPlaidConnector2ApplicationKt v1.1.0 using Java 17.0.10 with PID 1 (/workspace/BOOT-INF/classes started by cnb in /workspace)"
"2024-02-26T16:42:14.218Z DEBUG 1 --- [           main] .d.f.FireflyPlaidConnector2ApplicationKt : Running with Spring Boot v3.2.2, Spring v6.1.3"
"2024-02-26T16:42:14.219Z  INFO 1 --- [           main] .d.f.FireflyPlaidConnector2ApplicationKt : No active profile set, falling back to 1 default profile: ""default"""
"2024-02-26T16:42:32.419Z  INFO 1 --- [           main] .d.f.FireflyPlaidConnector2ApplicationKt : Started FireflyPlaidConnector2ApplicationKt in 21.503 seconds (process running for 24.695)"
"2024-02-26T16:42:35.716Z DEBUG 1 --- [           main] n.d.f.sync.PolledSyncRunner              : Beginning Plaid sync endpoint cursor initialization"
"2024-02-26T16:42:35.717Z DEBUG 1 --- [           main] n.d.f.sync.PolledSyncRunner              : Cursor map contains access-development-REDACTED, skipping initialization for it"
"2024-02-26T16:42:35.717Z DEBUG 1 --- [           main] n.d.f.sync.PolledSyncRunner              : Cursor map contains access-development-REDACTED, skipping initialization for it"
"2024-02-26T16:42:35.920Z DEBUG 1 --- [           main] n.d.f.sync.PolledSyncRunner              : Fetching page 0 of Firefly transactions with window starting at 2024-02-11"
"2024-02-26T16:42:40.225Z ERROR 1 --- [           main] o.s.boot.SpringApplication               : Application run failed"

"java.lang.reflect.UndeclaredThrowableException: Failed to invoke event listener method"
"HandlerMethod details: "
"Bean [net.djvk.fireflyPlaidConnector2.FireflyPlaidConnector2Application$$SpringCGLIB$$0]"
"Method [public void net.djvk.fireflyPlaidConnector2.FireflyPlaidConnector2Application.appReady()]"
"Resolved arguments: "

"   at org.springframework.context.event.ApplicationListenerMethodAdapter.doInvoke(ApplicationListenerMethodAdapter.java:382) ~[spring-context-6.1.3.jar:6.1.3]"
"   at org.springframework.context.event.ApplicationListenerMethodAdapter.processEvent(ApplicationListenerMethodAdapter.java:237) ~[spring-context-6.1.3.jar:6.1.3]"
"   at org.springframework.context.event.ApplicationListenerMethodAdapter.onApplicationEvent(ApplicationListenerMethodAdapter.java:168) ~[spring-context-6.1.3.jar:6.1.3]"
"   at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:178) ~[spring-context-6.1.3.jar:6.1.3]"
"   at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:171) ~[spring-context-6.1.3.jar:6.1.3]"
"   at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:149) ~[spring-context-6.1.3.jar:6.1.3]"
"   at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:451) ~[spring-context-6.1.3.jar:6.1.3]"
"   at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:384) ~[spring-context-6.1.3.jar:6.1.3]"
"   at org.springframework.boot.context.event.EventPublishingRunListener.ready(EventPublishingRunListener.java:109) ~[spring-boot-3.2.2.jar:3.2.2]"
"   at org.springframework.boot.SpringApplicationRunListeners.lambda$ready$6(SpringApplicationRunListeners.java:80) ~[spring-boot-3.2.2.jar:3.2.2]"
"   at java.base/java.lang.Iterable.forEach(Unknown Source) ~[na:na]"
"   at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:118) ~[spring-boot-3.2.2.jar:3.2.2]"
"   at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:112) ~[spring-boot-3.2.2.jar:3.2.2]"
"   at org.springframework.boot.SpringApplicationRunListeners.ready(SpringApplicationRunListeners.java:80) ~[spring-boot-3.2.2.jar:3.2.2]"
"   at org.springframework.boot.SpringApplication.run(SpringApplication.java:348) ~[spring-boot-3.2.2.jar:3.2.2]"
"   at org.springframework.boot.SpringApplication.run(SpringApplication.java:1354) ~[spring-boot-3.2.2.jar:3.2.2]"
"   at org.springframework.boot.SpringApplication.run(SpringApplication.java:1343) ~[spring-boot-3.2.2.jar:3.2.2]"
"   at net.djvk.fireflyPlaidConnector2.FireflyPlaidConnector2ApplicationKt.main(FireflyPlaidConnector2Application.kt:31) ~[classes/:1.1.0]"
"   at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]"
"   at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:na]"
"   at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:na]"
"   at java.base/java.lang.reflect.Method.invoke(Unknown Source) ~[na:na]"
"   at org.springframework.boot.loader.launch.Launcher.launch(Launcher.java:91) ~[workspace/:na]"
"   at org.springframework.boot.loader.launch.Launcher.launch(Launcher.java:53) ~[workspace/:na]"
"   at org.springframework.boot.loader.launch.JarLauncher.main(JarLauncher.java:58) ~[workspace/:na]"
"Caused by: io.ktor.serialization.JsonConvertException: Illegal json parameter found"
"   at io.ktor.serialization.jackson.JacksonConverter.deserialize(JacksonConverter.kt:67) ~[ktor-serialization-jackson-jvm-2.1.1.jar:2.1.1]"
"   at io.ktor.serialization.jackson.JacksonConverter$deserialize$1.invokeSuspend(JacksonConverter.kt) ~[ktor-serialization-jackson-jvm-2.1.1.jar:2.1.1]"
"   at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) ~[kotlin-stdlib-1.9.22.jar:1.9.22-release-704]"
"   at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106) ~[kotlinx-coroutines-core-jvm-1.7.3.jar:na]"
"   at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:280) ~[kotlinx-coroutines-core-jvm-1.7.3.jar:na]"
"   at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:85) ~[kotlinx-coroutines-core-jvm-1.7.3.jar:na]"
"   at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:59) ~[kotlinx-coroutines-core-jvm-1.7.3.jar:na]"
"   at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source) ~[kotlinx-coroutines-core-jvm-1.7.3.jar:na]"
"   at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:38) ~[kotlinx-coroutines-core-jvm-1.7.3.jar:na]"
"   at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source) ~[kotlinx-coroutines-core-jvm-1.7.3.jar:na]"
"   at net.djvk.fireflyPlaidConnector2.sync.PolledSyncRunner.run(PolledSyncRunner.kt:63) ~[classes/:1.1.0]"
"   at net.djvk.fireflyPlaidConnector2.FireflyPlaidConnector2Application.appReady(FireflyPlaidConnector2Application.kt:23) ~[classes/:1.1.0]"
"   at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]"
"   at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:na]"
"   at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:na]"
"   at java.base/java.lang.reflect.Method.invoke(Unknown Source) ~[na:na]"
"   at org.springframework.context.event.ApplicationListenerMethodAdapter.doInvoke(ApplicationListenerMethodAdapter.java:365) ~[spring-context-6.1.3.jar:6.1.3]"
"   ... 24 common frames omitted"
"Caused by: com.fasterxml.jackson.databind.exc.InvalidFormatException: Cannot deserialize value of type `java.lang.Integer` from String ""92.870000000000"": not a valid `java.lang.Integer` value"
" at [Source: (InputStreamReader); line: 1, column: 31280] (through reference chain: net.djvk.fireflyPlaidConnector2.api.firefly.models.TransactionArray[""data""]->java.util.ArrayList[17]->net.djvk.fireflyPlaidConnector2.api.firefly.models.TransactionRead[""attributes""]->net.djvk.fireflyPlaidConnector2.api.firefly.models.Transaction[""transactions""]->java.util.ArrayList[0]->net.djvk.fireflyPlaidConnector2.api.firefly.models.TransactionSplit[""foreign_amount""])"
"   at com.fasterxml.jackson.databind.exc.InvalidFormatException.from(InvalidFormatException.java:67) ~[jackson-databind-2.15.3.jar:2.15.3]"
"   at com.fasterxml.jackson.databind.DeserializationContext.weirdStringException(DeserializationContext.java:2002) ~[jackson-databind-2.15.3.jar:2.15.3]"
"   at com.fasterxml.jackson.databind.DeserializationContext.handleWeirdStringValue(DeserializationContext.java:1230) ~[jackson-databind-2.15.3.jar:2.15.3]"
"   at com.fasterxml.jackson.databind.deser.std.StdDeserializer._parseInteger(StdDeserializer.java:844) ~[jackson-databind-2.15.3.jar:2.15.3]"
"   at com.fasterxml.jackson.databind.deser.std.StdDeserializer._parseInteger(StdDeserializer.java:824) ~[jackson-databind-2.15.3.jar:2.15.3]"
"   at com.fasterxml.jackson.databind.deser.std.NumberDeserializers$IntegerDeserializer.deserialize(NumberDeserializers.java:531) ~[jackson-databind-2.15.3.jar:2.15.3]"
"   at com.fasterxml.jackson.databind.deser.std.NumberDeserializers$IntegerDeserializer.deserialize(NumberDeserializers.java:506) ~[jackson-databind-2.15.3.jar:2.15.3]"
"   at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:545) ~[jackson-databind-2.15.3.jar:2.15.3]"
"   at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:570) ~[jackson-databind-2.15.3.jar:2.15.3]"
"   at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:439) ~[jackson-databind-2.15.3.jar:2.15.3]"
"   at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1409) ~[jackson-databind-2.15.3.jar:2.15.3]"
"   at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:352) ~[jackson-databind-2.15.3.jar:2.15.3]"
"   at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:185) ~[jackson-databind-2.15.3.jar:2.15.3]"
"   at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer._deserializeFromArray(CollectionDeserializer.java:359) ~[jackson-databind-2.15.3.jar:2.15.3]"
"   at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:244) ~[jackson-databind-2.15.3.jar:2.15.3]"
"   at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:28) ~[jackson-databind-2.15.3.jar:2.15.3]"
"   at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:545) ~[jackson-databind-2.15.3.jar:2.15.3]"
"   at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:570) ~[jackson-databind-2.15.3.jar:2.15.3]"
"   at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:439) ~[jackson-databind-2.15.3.jar:2.15.3]"
"   at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1409) ~[jackson-databind-2.15.3.jar:2.15.3]"
"   at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:352) ~[jackson-databind-2.15.3.jar:2.15.3]"
"   at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:185) ~[jackson-databind-2.15.3.jar:2.15.3]"
"   at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:545) ~[jackson-databind-2.15.3.jar:2.15.3]"
"   at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:570) ~[jackson-databind-2.15.3.jar:2.15.3]"
"   at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:439) ~[jackson-databind-2.15.3.jar:2.15.3]"
"   at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1409) ~[jackson-databind-2.15.3.jar:2.15.3]"
"   at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:352) ~[jackson-databind-2.15.3.jar:2.15.3]"
"   at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:185) ~[jackson-databind-2.15.3.jar:2.15.3]"
"   at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer._deserializeFromArray(CollectionDeserializer.java:359) ~[jackson-databind-2.15.3.jar:2.15.3]"
"   at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:244) ~[jackson-databind-2.15.3.jar:2.15.3]"
"   at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:28) ~[jackson-databind-2.15.3.jar:2.15.3]"
"   at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:545) ~[jackson-databind-2.15.3.jar:2.15.3]"
"   at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:570) ~[jackson-databind-2.15.3.jar:2.15.3]"
"   at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:439) ~[jackson-databind-2.15.3.jar:2.15.3]"
"   at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1409) ~[jackson-databind-2.15.3.jar:2.15.3]"
"   at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:352) ~[jackson-databind-2.15.3.jar:2.15.3]"
"   at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:185) ~[jackson-databind-2.15.3.jar:2.15.3]"
"   at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:323) ~[jackson-databind-2.15.3.jar:2.15.3]"
"   at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4825) ~[jackson-databind-2.15.3.jar:2.15.3]"
"   at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3801) ~[jackson-databind-2.15.3.jar:2.15.3]"
"   at io.ktor.serialization.jackson.JacksonConverter$deserialize$2.invokeSuspend(JacksonConverter.kt:64) ~[ktor-serialization-jackson-jvm-2.1.1.jar:2.1.1]"
"   at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) ~[kotlin-stdlib-1.9.22.jar:1.9.22-release-704]"
"   at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108) ~[kotlinx-coroutines-core-jvm-1.7.3.jar:na]"
"   at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:115) ~[kotlinx-coroutines-core-jvm-1.7.3.jar:na]"
"   at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:103) ~[kotlinx-coroutines-core-jvm-1.7.3.jar:na]"
"   at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584) ~[kotlinx-coroutines-core-jvm-1.7.3.jar:na]"
"   at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793) ~[kotlinx-coroutines-core-jvm-1.7.3.jar:na]"
"   at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697) ~[kotlinx-coroutines-core-jvm-1.7.3.jar:na]"
"   at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684) ~[kotlinx-coroutines-core-jvm-1.7.3.jar:na]"

"2024-02-26T16:42:40.320Z  INFO 1 --- [           main] n.d.f.sync.PolledSyncRunner              : Shutting down class net.djvk.fireflyPlaidConnector2.sync.PolledSyncRunner"

The transaction mentioned in this particular error log is this one image

AmineI commented 6 months ago

I have updated the repro steps slightly as they were not precise enough

dvankley commented 5 months ago

Ah, I see, it's crapping out on the floating point conversion to integer. Derp. Knowing that and upon further reflection, I'm going to say let's go ahead and merge your original PR. It fixes the problem (as you said) and the actual value of my change https://github.com/dvankley/firefly-plaid-connector-2/commit/08fac2bfcf6f201ddef912f42239c6730f210fc7 was removing the obsolete "0" literals, not the String->Int type changes. I would like for currency amounts to be represented in the connector with a more suitable data type, but that's a larger undertaking that doesn't seem worth the trouble right now.

dvankley commented 5 months ago

I'll cut a new release soonish after I merge in dependabot updates and do my basic regression tests.

dvankley commented 5 months ago

Should be fixed in https://github.com/dvankley/firefly-plaid-connector-2/releases/tag/v1.1.1