[destination-duckdb] Airbyte throwing error`Cannot invoke "io.airbyte.protocol.models.AirbyteGlobalState.getStreamStates()"` for CDC sync

[g-diwakar]

Connector Name

destination-duckdb

Connector Version

0.3.5

What step the error happened?

During the sync

Relevant information

Use the latest version of open source airbyte. Use mysql or postgres as source connector for cdc incremental load. Run the sync and after some minutes (depends on your data) the replication will stop throwing following error.

I have attached log file here.

default_workspace_job_9_attempt_5_txt.txt

Relevant log output

2024-05-29 04:14:18 platform > readFromDestination: exception caught
java.lang.NullPointerException: Cannot invoke "io.airbyte.protocol.models.AirbyteGlobalState.getStreamStates()" because the return value of "io.airbyte.protocol.models.AirbyteStateMessage.getGlobal()" is null
        at io.airbyte.workers.internal.bookkeeping.ParallelStreamStatsTracker.updateDestinationStateStats(ParallelStreamStatsTracker.kt:144) ~[io.airbyte-airbyte-commons-worker-0.61.0.jar:?]
        at io.airbyte.workers.internal.syncpersistence.SyncPersistenceImpl.updateDestinationStateStats(SyncPersistence.kt:356) ~[io.airbyte-airbyte-commons-worker-0.61.0.jar:?]
        at io.airbyte.workers.internal.bookkeeping.AirbyteMessageTracker.acceptFromDestination(AirbyteMessageTracker.kt:66) ~[io.airbyte-airbyte-commons-worker-0.61.0.jar:?]
        at io.airbyte.workers.general.ReplicationWorkerHelper.internalProcessMessageFromDestination(ReplicationWorkerHelper.kt:415) ~[io.airbyte-airbyte-commons-worker-0.61.0.jar:?]
        at io.airbyte.workers.general.ReplicationWorkerHelper.processMessageFromDestination(ReplicationWorkerHelper.kt:303) ~[io.airbyte-airbyte-commons-worker-0.61.0.jar:?]
        at io.airbyte.workers.general.BufferedReplicationWorker.readFromDestination(BufferedReplicationWorker.java:488) ~[io.airbyte-airbyte-commons-worker-0.61.0.jar:?]
        at io.airbyte.workers.general.BufferedReplicationWorker.lambda$runAsync$2(BufferedReplicationWorker.java:235) ~[io.airbyte-airbyte-commons-worker-0.61.0.jar:?]
        at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804) ~[?:?]
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
        at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
2024-05-29 04:14:18 platform > readFromDestination: done. (writeToDestFailed:false, dest.isFinished:false)
2024-05-29 04:14:18 platform > thread status... timeout thread: false , replication thread: true

Contribute

• [ ] Yes, I want to contribute

[marcosmarxm]

Hello @g-diwakar could you try another destination? If you disable CDC it works?

[g-diwakar]

Sorry I didn't update here. It works with other destinations. Perhaps the issue is within the duckdb connector.

[N-liraj-khanna]

This is still not fixed, even for weaviate not working