search

airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
16.27k stars 4.15k forks source link

[source-mongodb] Records get extracted and loaded but it only shows extracted and 0 loaded #45133

Open Yatindra29 opened 2 months ago

Yatindra29 commented 2 months ago

Connector Name

source-mongodb-v2

Connector Version

1.5.9

What step the error happened?

During the sync

Relevant information

replication-orchestrator > failures: [ { "failureOrigin" : "replication", "internalMessage" : "Cannot invoke \"io.airbyte.protocol.models.AirbyteGlobalState.getStreamStates()\" because the return value of \"io.airbyte.protocol.models.AirbyteStateMessage.getGlobal()\" is null", "externalMessage" : "Something went wrong during replication", "metadata" : { "attemptNumber" : 3, "jobId" : 14 }, "stacktrace" : "java.lang.NullPointerException: Cannot invoke \"io.airbyte.protocol.models.AirbyteGlobalState.getStreamStates()\" because the return value of \"io.airbyte.protocol.models.AirbyteStateMessage.getGlobal()\" is null

This is the error that I am getting on creating a connection between MongoDB and Weaviate. It extracts all data from MongoDB, so it shows data extracted as it is supposed to, but it loads data in Weaviate as per the query done using GraphQL but it shows 0 records loaded and the error above. Please help me with this.

Relevant log output

2024-09-03 10:30:24 platform > Retry State: RetryManager(completeFailureBackoffPolicy=BackoffPolicy(minInterval=PT10S, maxInterval=PT30M, base=3), partialFailureBackoffPolicy=null, successiveCompleteFailureLimit=5, totalCompleteFailureLimit=10, successivePartialFailureLimit=1000, totalPartialFailureLimit=20, successiveCompleteFailures=3, totalCompleteFailures=3, successivePartialFailures=0, totalPartialFailures=0)
2024-09-03 10:30:24 platform > Backing off for: 1 minute 30 seconds.
2024-09-03 10:31:55 INFO i.a.w.l.p.s.m.Stage(apply):39 - APPLY Stage: CLAIM — (workloadId = b2e713cd-cc36-4c0a-b5bd-b47cb8a0561e_14_3_check) — (dataplaneId = local)
2024-09-03 10:32:06 INFO i.a.c.i.LineGobbler(voidCall):166 - 
2024-09-03 10:32:06 INFO i.a.c.ConnectorWatcher(run):87 - Connector exited, processing output
2024-09-03 10:32:07 INFO i.a.c.ConnectorWatcher(run):90 - Output file jobOutput.json found
2024-09-03 10:32:07 INFO i.a.c.i.LineGobbler(voidCall):166 - ----- START CHECK -----
2024-09-03 10:32:07 INFO i.a.c.ConnectorWatcher(run):96 - Connector exited with 0
2024-09-03 10:32:07 INFO i.a.c.i.LineGobbler(voidCall):166 - 
2024-09-03 10:32:07 INFO i.a.w.i.VersionedAirbyteStreamFactory(create):189 - Reading messages from protocol version 0.2.0
2024-09-03 10:32:07 WARN i.a.m.l.MetricClientFactory(getMetricClient):43 - MetricClient has not been initialized. Must call MetricClientFactory.CreateMetricClient before using MetricClient. Using a dummy client for now. Ignore this if Airbyte is configured to not publish any metrics.
2024-09-03 10:32:07 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):308 - INFO main i.a.i.s.m.MongoDbSource(main):52 starting source: class io.airbyte.integrations.source.mongodb.MongoDbSource
2024-09-03 10:32:07 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):308 - INFO main i.a.c.i.b.IntegrationCliParser$Companion(parseOptions):144 integration args: {check=null, config=/config/connectionConfiguration.json}
2024-09-03 10:32:07 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):308 - INFO main i.a.c.i.b.IntegrationRunner(runInternal):130 Running integration: io.airbyte.integrations.source.mongodb.MongoDbSource
2024-09-03 10:32:07 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):308 - INFO main i.a.c.i.b.IntegrationRunner(runInternal):131 Command: CHECK
2024-09-03 10:32:07 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):308 - INFO main i.a.c.i.b.IntegrationRunner(runInternal):132 Integration config: IntegrationConfig{command=CHECK, configPath='/config/connectionConfiguration.json', catalogPath='null', statePath='null'}
2024-09-03 10:32:07 WARN i.a.w.i.VersionedAirbyteStreamFactory(internalLog):305 - WARN main c.n.s.JsonMetaSchema(newValidator):278 Unknown keyword groups - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2024-09-03 10:32:07 WARN i.a.w.i.VersionedAirbyteStreamFactory(internalLog):305 - WARN main c.n.s.JsonMetaSchema(newValidator):278 Unknown keyword order - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2024-09-03 10:32:07 WARN i.a.w.i.VersionedAirbyteStreamFactory(internalLog):305 - WARN main c.n.s.JsonMetaSchema(newValidator):278 Unknown keyword group - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2024-09-03 10:32:07 WARN i.a.w.i.VersionedAirbyteStreamFactory(internalLog):305 - WARN main c.n.s.JsonMetaSchema(newValidator):278 Unknown keyword display_type - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2024-09-03 10:32:07 WARN i.a.w.i.VersionedAirbyteStreamFactory(internalLog):305 - WARN main c.n.s.JsonMetaSchema(newValidator):278 Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2024-09-03 10:32:07 WARN i.a.w.i.VersionedAirbyteStreamFactory(internalLog):305 - WARN main c.n.s.JsonMetaSchema(newValidator):278 Unknown keyword always_show - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2024-09-03 10:32:07 WARN i.a.w.i.VersionedAirbyteStreamFactory(internalLog):305 - WARN main c.n.s.JsonMetaSchema(newValidator):278 Unknown keyword min - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2024-09-03 10:32:07 WARN i.a.w.i.VersionedAirbyteStreamFactory(internalLog):305 - WARN main c.n.s.JsonMetaSchema(newValidator):278 Unknown keyword max - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2024-09-03 10:32:07 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):308 - INFO main c.m.i.d.l.SLF4JLogger(info):71 MongoClient with metadata {"driver": {"name": "mongo-java-driver|sync|Airbyte", "version": "4.11.0"}, "os": {"type": "Linux", "name": "Linux", "architecture": "amd64", "version": "5.15.153.1-microsoft-standard-WSL2"}, "platform": "Java/Amazon.com Inc./21.0.4+7-LTS"} created with settings MongoClientSettings{readPreference=ReadPreference{name=secondaryPreferred, hedgeOptions=null}, writeConcern=WriteConcern{w=null, wTimeout=null ms, journal=null}, retryWrites=true, retryReads=true, readConcern=ReadConcern{level=null}, credential=MongoCredential{mechanism=null, userName='READ_ONLY_USER', source='admin', password=<hidden>, mechanismProperties=<hidden>}, transportSettings=null, streamFactoryFactory=null, commandListeners=[], codecRegistry=ProvidersCodecRegistry{codecProviders=[ValueCodecProvider{}, BsonValueCodecProvider{}, DBRefCodecProvider{}, DBObjectCodecProvider{}, DocumentCodecProvider{}, CollectionCodecProvider{}, IterableCodecProvider{}, MapCodecProvider{}, GeoJsonCodecProvider{}, GridFSFileCodecProvider{}, Jsr310CodecProvider{}, JsonObjectCodecProvider{}, BsonCodecProvider{}, EnumCodecProvider{}, com.mongodb.client.model.mql.ExpressionCodecProvider@5300f14a, com.mongodb.Jep395RecordCodecProvider@1f86099a, com.mongodb.KotlinCodecProvider@77bb0ab5]}, loggerSettings=LoggerSettings{maxDocumentLength=1000}, clusterSettings={hosts=[127.0.0.1:27017], srvHost=cluster0.jwvtw.mongodb.net, srvServiceName=mongodb, mode=MULTIPLE, requiredClusterType=REPLICA_SET, requiredReplicaSetName='atlas-3fnexm-shard-0', serverSelector='null', clusterListeners='[]', serverSelectionTimeout='30000 ms', localThreshold='15 ms'}, socketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=0, receiveBufferSize=0, proxySettings=ProxySettings{host=null, port=null, username=null, password=null}}, heartbeatSocketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=10000, receiveBufferSize=0, proxySettings=ProxySettings{host=null, port=null, username=null, password=null}}, connectionPoolSettings=ConnectionPoolSettings{maxSize=100, minSize=0, maxWaitTimeMS=120000, maxConnectionLifeTimeMS=0, maxConnectionIdleTimeMS=0, maintenanceInitialDelayMS=0, maintenanceFrequencyMS=60000, connectionPoolListeners=[], maxConnecting=2}, serverSettings=ServerSettings{heartbeatFrequencyMS=10000, minHeartbeatFrequencyMS=500, serverListeners='[]', serverMonitorListeners='[]'}, sslSettings=SslSettings{enabled=true, invalidHostNameAllowed=false, context=null}, applicationName='null', compressorList=[], uuidRepresentation=UNSPECIFIED, serverApi=null, autoEncryptionSettings=null, dnsClient=null, inetAddressResolver=null, contextProvider=null}
2024-09-03 10:32:07 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):308 - INFO main c.m.i.d.l.SLF4JLogger(info):71 No server chosen by ReadPreferenceServerSelector{readPreference=primary} from cluster description ClusterDescription{type=UNKNOWN, connectionMode=MULTIPLE, serverDescriptions=[]}. Waiting for 30000 ms before timing out
2024-09-03 10:32:07 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):308 - INFO cluster-ClusterId{value='66d6e5a1234d696a52747e9b', description='null'}-srv-cluster0.jwvtw.mongodb.net c.m.i.d.l.SLF4JLogger(info):71 Adding discovered server cluster0-shard-00-00.jwvtw.mongodb.net:27017 to client view of cluster
2024-09-03 10:32:07 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):308 - INFO cluster-ClusterId{value='66d6e5a1234d696a52747e9b', description='null'}-srv-cluster0.jwvtw.mongodb.net c.m.i.d.l.SLF4JLogger(info):71 Adding discovered server cluster0-shard-00-01.jwvtw.mongodb.net:27017 to client view of cluster
2024-09-03 10:32:07 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):308 - INFO cluster-ClusterId{value='66d6e5a1234d696a52747e9b', description='null'}-srv-cluster0.jwvtw.mongodb.net c.m.i.d.l.SLF4JLogger(info):71 Adding discovered server cluster0-shard-00-02.jwvtw.mongodb.net:27017 to client view of cluster
2024-09-03 10:32:07 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):308 - INFO cluster-ClusterId{value='66d6e5a1234d696a52747e9b', description='null'}-cluster0-shard-00-02.jwvtw.mongodb.net:27017 c.m.i.d.l.SLF4JLogger(info):71 Monitor thread successfully connected to server with description ServerDescription{address=cluster0-shard-00-02.jwvtw.mongodb.net:27017, type=REPLICA_SET_SECONDARY, state=CONNECTED, ok=true, minWireVersion=0, maxWireVersion=21, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=412872014, setName='atlas-3fnexm-shard-0', canonicalAddress=cluster0-shard-00-02.jwvtw.mongodb.net:27017, hosts=[cluster0-shard-00-00.jwvtw.mongodb.net:27017, cluster0-shard-00-01.jwvtw.mongodb.net:27017, cluster0-shard-00-02.jwvtw.mongodb.net:27017], passives=[], arbiters=[], primary='cluster0-shard-00-01.jwvtw.mongodb.net:27017', tagSet=TagSet{[Tag{name='availabilityZone', value='aps1-az2'}, Tag{name='diskState', value='READY'}, Tag{name='nodeType', value='ELECTABLE'}, Tag{name='provider', value='AWS'}, Tag{name='region', value='AP_SOUTH_1'}, Tag{name='workloadType', value='OPERATIONAL'}]}, electionId=null, setVersion=92, topologyVersion=TopologyVersion{processId=66cc3f7e08f6df3938bbe157, counter=5}, lastWriteDate=Tue Sep 03 10:32:02 UTC 2024, lastUpdateTimeNanos=10359225123736}
2024-09-03 10:32:07 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):308 - INFO cluster-ClusterId{value='66d6e5a1234d696a52747e9b', description='null'}-cluster0-shard-00-00.jwvtw.mongodb.net:27017 c.m.i.d.l.SLF4JLogger(info):71 Monitor thread successfully connected to server with description ServerDescription{address=cluster0-shard-00-00.jwvtw.mongodb.net:27017, type=REPLICA_SET_SECONDARY, state=CONNECTED, ok=true, minWireVersion=0, maxWireVersion=21, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=412861107, setName='atlas-3fnexm-shard-0', canonicalAddress=cluster0-shard-00-00.jwvtw.mongodb.net:27017, hosts=[cluster0-shard-00-00.jwvtw.mongodb.net:27017, cluster0-shard-00-01.jwvtw.mongodb.net:27017, cluster0-shard-00-02.jwvtw.mongodb.net:27017], passives=[], arbiters=[], primary='cluster0-shard-00-01.jwvtw.mongodb.net:27017', tagSet=TagSet{[Tag{name='availabilityZone', value='aps1-az1'}, Tag{name='diskState', value='READY'}, Tag{name='nodeType', value='ELECTABLE'}, Tag{name='provider', value='AWS'}, Tag{name='region', value='AP_SOUTH_1'}, Tag{name='workloadType', value='OPERATIONAL'}]}, electionId=null, setVersion=92, topologyVersion=TopologyVersion{processId=66cc3e35804a86ace670d69e, counter=5}, lastWriteDate=Tue Sep 03 10:32:02 UTC 2024, lastUpdateTimeNanos=10359225270190}
2024-09-03 10:32:07 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):308 - INFO cluster-ClusterId{value='66d6e5a1234d696a52747e9b', description='null'}-cluster0-shard-00-01.jwvtw.mongodb.net:27017 c.m.i.d.l.SLF4JLogger(info):71 Monitor thread successfully connected to server with description ServerDescription{address=cluster0-shard-00-01.jwvtw.mongodb.net:27017, type=REPLICA_SET_PRIMARY, state=CONNECTED, ok=true, minWireVersion=0, maxWireVersion=21, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=409211956, setName='atlas-3fnexm-shard-0', canonicalAddress=cluster0-shard-00-01.jwvtw.mongodb.net:27017, hosts=[cluster0-shard-00-00.jwvtw.mongodb.net:27017, cluster0-shard-00-01.jwvtw.mongodb.net:27017, cluster0-shard-00-02.jwvtw.mongodb.net:27017], passives=[], arbiters=[], primary='cluster0-shard-00-01.jwvtw.mongodb.net:27017', tagSet=TagSet{[Tag{name='availabilityZone', value='aps1-az3'}, Tag{name='diskState', value='READY'}, Tag{name='nodeType', value='ELECTABLE'}, Tag{name='provider', value='AWS'}, Tag{name='region', value='AP_SOUTH_1'}, Tag{name='workloadType', value='OPERATIONAL'}]}, electionId=7fffffff000000000000005f, setVersion=92, topologyVersion=TopologyVersion{processId=66cc3d11432d320fa42dc774, counter=9}, lastWriteDate=Tue Sep 03 10:32:02 UTC 2024, lastUpdateTimeNanos=10359226376179}
2024-09-03 10:32:07 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):308 - INFO cluster-ClusterId{value='66d6e5a1234d696a52747e9b', description='null'}-cluster0-shard-00-01.jwvtw.mongodb.net:27017 c.m.i.d.l.SLF4JLogger(info):71 Discovered replica set primary cluster0-shard-00-01.jwvtw.mongodb.net:27017 with max election id 7fffffff000000000000005f and max set version 92
2024-09-03 10:32:07 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):308 - INFO main i.a.i.s.m.MongoDbSource(check):97 The source passed the check operation test!
2024-09-03 10:32:07 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):308 - INFO main i.a.c.i.b.IntegrationRunner(runInternal):224 Completed integration: io.airbyte.integrations.source.mongodb.MongoDbSource
2024-09-03 10:32:07 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):308 - INFO main i.a.i.s.m.MongoDbSource(main):54 completed source: class io.airbyte.integrations.source.mongodb.MongoDbSource
2024-09-03 10:32:07 INFO i.a.c.ConnectorMessageProcessor(updateConfigFromControlMessage):231 - Checking for optional control message...
2024-09-03 10:32:08 INFO i.a.c.ConnectorWatcher(run):134 - Writing output of b2e713cd-cc36-4c0a-b5bd-b47cb8a0561e_14_3_check to the doc store
2024-09-03 10:32:09 INFO i.a.c.ConnectorWatcher(run):136 - Marking workload as successful
2024-09-03 10:32:09 INFO i.a.c.i.LineGobbler(voidCall):166 - 
2024-09-03 10:32:09 INFO i.a.c.ConnectorWatcher(exitProperly):189 - Deliberately exiting process with code 0.
2024-09-03 10:32:09 INFO i.a.c.i.LineGobbler(voidCall):166 - ----- END CHECK -----
2024-09-03 10:32:09 INFO i.a.c.i.LineGobbler(voidCall):166 - 
2024-09-03 10:32:19 INFO i.a.c.i.LineGobbler(voidCall):166 - 
2024-09-03 10:32:19 INFO i.a.c.i.LineGobbler(voidCall):166 - ----- START CHECK -----
2024-09-03 10:32:19 INFO i.a.c.i.LineGobbler(voidCall):166 - 
2024-09-03 10:32:25 INFO i.a.c.ConnectorWatcher(run):87 - Connector exited, processing output
2024-09-03 10:32:25 INFO i.a.c.ConnectorWatcher(run):90 - Output file jobOutput.json found
2024-09-03 10:32:25 INFO i.a.c.ConnectorWatcher(run):96 - Connector exited with 0
2024-09-03 10:32:25 INFO i.a.w.i.VersionedAirbyteStreamFactory(create):189 - Reading messages from protocol version 0.2.0
2024-09-03 10:32:25 WARN i.a.m.l.MetricClientFactory(getMetricClient):43 - MetricClient has not been initialized. Must call MetricClientFactory.CreateMetricClient before using MetricClient. Using a dummy client for now. Ignore this if Airbyte is configured to not publish any metrics.
2024-09-03 10:32:25 INFO i.a.c.ConnectorMessageProcessor(updateConfigFromControlMessage):231 - Checking for optional control message...
2024-09-03 10:32:25 INFO i.a.c.ConnectorWatcher(run):134 - Writing output of 7b7d7a0d-954c-45a0-bcfc-39a634b97736_14_3_check to the doc store
2024-09-03 10:32:25 INFO i.a.c.ConnectorWatcher(run):136 - Marking workload as successful
2024-09-03 10:32:25 INFO i.a.c.ConnectorWatcher(exitProperly):189 - Deliberately exiting process with code 0.
2024-09-03 10:32:25 INFO i.a.c.i.LineGobbler(voidCall):166 - 
2024-09-03 10:32:25 INFO i.a.c.i.LineGobbler(voidCall):166 - ----- END CHECK -----
2024-09-03 10:32:25 INFO i.a.c.i.LineGobbler(voidCall):166 - 
2024-09-03 10:32:35 INFO i.m.r.Micronaut(start):101 - Startup completed in 3233ms. Server Running: http://orchestrator-repl-job-14-attempt-3:9000
2024-09-03 10:32:38 replication-orchestrator > Unknown keyword airbyte_type - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword or if it should generate annotations AnnotationKeyword
2024-09-03 10:31:55 INFO i.a.w.l.c.WorkloadApiClient(claim):75 - Claimed: true for b2e713cd-cc36-4c0a-b5bd-b47cb8a0561e_14_3_check via API for local
2024-09-03 10:31:55 INFO i.a.w.l.p.s.m.Stage(apply):39 - APPLY Stage: CHECK_STATUS — (workloadId = b2e713cd-cc36-4c0a-b5bd-b47cb8a0561e_14_3_check) — (dataplaneId = local)
2024-09-03 10:31:55 INFO i.a.w.l.p.s.CheckStatusStage(applyStage):59 - No pod found running for workload b2e713cd-cc36-4c0a-b5bd-b47cb8a0561e_14_3_check
2024-09-03 10:31:55 INFO i.a.w.l.p.s.m.Stage(apply):39 - APPLY Stage: BUILD — (workloadId = b2e713cd-cc36-4c0a-b5bd-b47cb8a0561e_14_3_check) — (dataplaneId = local)
2024-09-03 10:31:55 INFO i.a.w.l.p.s.m.Stage(apply):39 - APPLY Stage: MUTEX — (workloadId = b2e713cd-cc36-4c0a-b5bd-b47cb8a0561e_14_3_check) — (dataplaneId = local)
2024-09-03 10:31:55 INFO i.a.w.l.p.s.EnforceMutexStage(applyStage):50 - No mutex key specified for workload: b2e713cd-cc36-4c0a-b5bd-b47cb8a0561e_14_3_check. Continuing...
2024-09-03 10:31:55 INFO i.a.w.l.p.s.m.Stage(apply):39 - APPLY Stage: LAUNCH — (workloadId = b2e713cd-cc36-4c0a-b5bd-b47cb8a0561e_14_3_check) — (dataplaneId = local)
2024-09-03 10:31:59 INFO i.a.w.l.c.WorkloadApiClient(updateStatusToLaunched):60 - Attempting to update workload: b2e713cd-cc36-4c0a-b5bd-b47cb8a0561e_14_3_check to LAUNCHED.
2024-09-03 10:31:59 INFO i.a.w.l.p.h.SuccessHandler(accept):60 - Pipeline completed for workload: b2e713cd-cc36-4c0a-b5bd-b47cb8a0561e_14_3_check.
2024-09-03 10:32:10 INFO i.a.w.l.p.s.m.Stage(apply):39 - APPLY Stage: CLAIM — (workloadId = 7b7d7a0d-954c-45a0-bcfc-39a634b97736_14_3_check) — (dataplaneId = local)
2024-09-03 10:32:10 INFO i.a.w.l.c.WorkloadApiClient(claim):75 - Claimed: true for 7b7d7a0d-954c-45a0-bcfc-39a634b97736_14_3_check via API for local
2024-09-03 10:32:10 INFO i.a.w.l.p.s.m.Stage(apply):39 - APPLY Stage: CHECK_STATUS — (workloadId = 7b7d7a0d-954c-45a0-bcfc-39a634b97736_14_3_check) — (dataplaneId = local)
2024-09-03 10:32:10 INFO i.a.w.l.p.s.CheckStatusStage(applyStage):59 - No pod found running for workload 7b7d7a0d-954c-45a0-bcfc-39a634b97736_14_3_check
2024-09-03 10:32:10 INFO i.a.w.l.p.s.m.Stage(apply):39 - APPLY Stage: BUILD — (workloadId = 7b7d7a0d-954c-45a0-bcfc-39a634b97736_14_3_check) — (dataplaneId = local)
2024-09-03 10:32:10 INFO i.a.w.l.p.s.m.Stage(apply):39 - APPLY Stage: MUTEX — (workloadId = 7b7d7a0d-954c-45a0-bcfc-39a634b97736_14_3_check) — (dataplaneId = local)
2024-09-03 10:32:10 INFO i.a.w.l.p.s.EnforceMutexStage(applyStage):50 - No mutex key specified for workload: 7b7d7a0d-954c-45a0-bcfc-39a634b97736_14_3_check. Continuing...
2024-09-03 10:32:10 INFO i.a.w.l.p.s.m.Stage(apply):39 - APPLY Stage: LAUNCH — (workloadId = 7b7d7a0d-954c-45a0-bcfc-39a634b97736_14_3_check) — (dataplaneId = local)
2024-09-03 10:32:15 INFO i.a.w.l.c.WorkloadApiClient(updateStatusToLaunched):60 - Attempting to update workload: 7b7d7a0d-954c-45a0-bcfc-39a634b97736_14_3_check to LAUNCHED.
2024-09-03 10:32:15 INFO i.a.w.l.p.h.SuccessHandler(accept):60 - Pipeline completed for workload: 7b7d7a0d-954c-45a0-bcfc-39a634b97736_14_3_check.
2024-09-03 10:32:27 INFO i.a.w.l.p.s.m.Stage(apply):39 - APPLY Stage: CLAIM — (workloadId = 4263d587-4ae4-4995-b0d1-7e970c9b1322_14_3_sync) — (dataplaneId = local)
2024-09-03 10:32:27 INFO i.a.w.l.c.WorkloadApiClient(claim):75 - Claimed: true for 4263d587-4ae4-4995-b0d1-7e970c9b1322_14_3_sync via API for local
2024-09-03 10:32:27 INFO i.a.w.l.p.s.m.Stage(apply):39 - APPLY Stage: CHECK_STATUS — (workloadId = 4263d587-4ae4-4995-b0d1-7e970c9b1322_14_3_sync) — (dataplaneId = local)
2024-09-03 10:32:27 INFO i.a.w.l.p.s.CheckStatusStage(applyStage):59 - No pod found running for workload 4263d587-4ae4-4995-b0d1-7e970c9b1322_14_3_sync
2024-09-03 10:32:27 INFO i.a.w.l.p.s.m.Stage(apply):39 - APPLY Stage: BUILD — (workloadId = 4263d587-4ae4-4995-b0d1-7e970c9b1322_14_3_sync) — (dataplaneId = local)
2024-09-03 10:32:27 INFO i.a.w.l.p.s.m.Stage(apply):39 - APPLY Stage: MUTEX — (workloadId = 4263d587-4ae4-4995-b0d1-7e970c9b1322_14_3_sync) — (dataplaneId = local)
2024-09-03 10:32:27 INFO i.a.w.l.p.s.EnforceMutexStage(applyStage):54 - Mutex key: 4263d587-4ae4-4995-b0d1-7e970c9b1322 specified for workload: 4263d587-4ae4-4995-b0d1-7e970c9b1322_14_3_sync. Attempting to delete existing pods...
2024-09-03 10:32:27 INFO i.a.w.l.p.s.EnforceMutexStage(applyStage):65 - Mutex key: 4263d587-4ae4-4995-b0d1-7e970c9b1322 specified for workload: 4263d587-4ae4-4995-b0d1-7e970c9b1322_14_3_sync found no existing pods. Continuing...
2024-09-03 10:32:27 INFO i.a.w.l.p.s.m.Stage(apply):39 - APPLY Stage: LAUNCH — (workloadId = 4263d587-4ae4-4995-b0d1-7e970c9b1322_14_3_sync) — (dataplaneId = local)
2024-09-03 10:32:27 INFO i.a.c.EnvConfigs(getEnvOrDefault):342 - Using default value for environment variable SIDECAR_KUBE_CPU_LIMIT: '2.0'
2024-09-03 10:32:27 INFO i.a.c.EnvConfigs(getEnvOrDefault):342 - Using default value for environment variable SOCAT_KUBE_CPU_LIMIT: '2.0'
2024-09-03 10:32:27 INFO i.a.c.EnvConfigs(getEnvOrDefault):342 - Using default value for environment variable SIDECAR_KUBE_CPU_REQUEST: '0.1'
2024-09-03 10:32:27 INFO i.a.c.EnvConfigs(getEnvOrDefault):342 - Using default value for environment variable SOCAT_KUBE_CPU_REQUEST: '0.1'
2024-09-03 10:32:42 INFO i.a.w.l.c.WorkloadApiClient(updateStatusToLaunched):60 - Attempting to update workload: 4263d587-4ae4-4995-b0d1-7e970c9b1322_14_3_sync to LAUNCHED.
2024-09-03 10:32:42 INFO i.a.w.l.p.h.SuccessHandler(accept):60 - Pipeline completed for workload: 4263d587-4ae4-4995-b0d1-7e970c9b1322_14_3_sync.
2024-09-03 10:32:27 platform > Using job log path: /workspace/14/3/logs.log
2024-09-03 10:32:27 platform > Executing worker wrapper. Airbyte version: 0.64.1
2024-09-03 10:32:27 platform > Creating workload 4263d587-4ae4-4995-b0d1-7e970c9b1322_14_3_sync
2024-09-03 10:32:27 platform > Workload 4263d587-4ae4-4995-b0d1-7e970c9b1322_14_3_sync is pending
2024-09-03 10:32:36 INFO i.a.f.ConfigFileClient(<init>):113 - path /flags does not exist, will return default flag values
2024-09-03 10:32:36 WARN i.a.m.l.MetricClientFactory(initialize):72 - MetricClient was not recognized or not provided. Accepted values are `datadog` or `otel`. 
2024-09-03 10:33:28 INFO i.a.a.SegmentAnalyticsClient(close):233 - Closing Segment analytics client...
2024-09-03 10:33:28 INFO i.a.a.BlockingShutdownAnalyticsPlugin(waitForFlush):289 - Waiting for Segment analytic client to flush enqueued messages...
2024-09-03 10:33:28 INFO i.a.a.BlockingShutdownAnalyticsPlugin(waitForFlush):301 - Segment analytic client flush complete.
2024-09-03 10:33:28 INFO i.a.a.SegmentAnalyticsClient(close):237 - Segment analytics client closed.  No new events will be accepted.
2024-09-03 10:32:38 replication-orchestrator > Concurrent stream read enabled? false
2024-09-03 10:32:38 replication-orchestrator > Setting up source...
2024-09-03 10:32:38 replication-orchestrator > Using default value for environment variable SIDECAR_KUBE_MEMORY_LIMIT: '50Mi'
2024-09-03 10:32:38 replication-orchestrator > Using default value for environment variable SIDECAR_MEMORY_REQUEST: '25Mi'
2024-09-03 10:32:38 replication-orchestrator > Using default value for environment variable SIDECAR_KUBE_CPU_LIMIT: '2.0'
2024-09-03 10:32:38 replication-orchestrator > Using default value for environment variable SIDECAR_KUBE_CPU_REQUEST: '0.1'
2024-09-03 10:32:38 replication-orchestrator > Using default value for environment variable SIDECAR_KUBE_MEMORY_LIMIT: '50Mi'
2024-09-03 10:32:38 replication-orchestrator > Using default value for environment variable SIDECAR_MEMORY_REQUEST: '25Mi'
2024-09-03 10:32:38 replication-orchestrator > Using default value for environment variable SIDECAR_KUBE_CPU_LIMIT: '2.0'
2024-09-03 10:32:38 replication-orchestrator > Using default value for environment variable SIDECAR_KUBE_CPU_REQUEST: '0.1'
2024-09-03 10:32:38 replication-orchestrator > Using default value for environment variable SIDECAR_KUBE_MEMORY_LIMIT: '50Mi'
2024-09-03 10:32:38 replication-orchestrator > Using default value for environment variable SIDECAR_MEMORY_REQUEST: '25Mi'
2024-09-03 10:32:38 replication-orchestrator > Setting up destination...
2024-09-03 10:32:38 replication-orchestrator > Setting up replication worker...
2024-09-03 10:32:38 replication-orchestrator > starting state flush thread for connectionId 4263d587-4ae4-4995-b0d1-7e970c9b1322
2024-09-03 10:32:38 replication-orchestrator > Using ClosableLinkedBlockingQueue
2024-09-03 10:32:38 replication-orchestrator > Using ClosableLinkedBlockingQueue
2024-09-03 10:32:38 replication-orchestrator > Running replication worker...
2024-09-03 10:32:38 replication-orchestrator > start sync worker. job id: 14 attempt id: 3
2024-09-03 10:32:38 replication-orchestrator > 
2024-09-03 10:32:38 replication-orchestrator > ----- START REPLICATION -----
2024-09-03 10:32:38 replication-orchestrator > 
2024-09-03 10:32:38 replication-orchestrator > Running destination...
2024-09-03 10:32:38 replication-orchestrator > Using default value for environment variable SIDECAR_KUBE_CPU_LIMIT: '2.0'
2024-09-03 10:32:38 replication-orchestrator > Using default value for environment variable SIDECAR_KUBE_CPU_LIMIT: '2.0'
2024-09-03 10:32:38 replication-orchestrator > Using default value for environment variable SIDECAR_KUBE_CPU_REQUEST: '0.1'
2024-09-03 10:32:38 replication-orchestrator > Using default value for environment variable SIDECAR_KUBE_CPU_REQUEST: '0.1'
2024-09-03 10:32:38 replication-orchestrator > Attempting to start pod = source-mongodb-v2-read-14-3-wxtaw for airbyte/source-mongodb-v2:1.5.9 with resources ConnectorResourceRequirements[main=io.airbyte.config.ResourceRequirements@755ba8fd[cpuRequest=1,cpuLimit=3,memoryRequest=2Gi,memoryLimit=4Gi,additionalProperties={}], heartbeat=io.airbyte.config.ResourceRequirements@1d8a78d8[cpuRequest=0.05,cpuLimit=0.2,memoryRequest=25Mi,memoryLimit=50Mi,additionalProperties={}], stdErr=io.airbyte.config.ResourceRequirements@138e6434[cpuRequest=0.01,cpuLimit=0.5,memoryRequest=25Mi,memoryLimit=50Mi,additionalProperties={}], stdIn=null, stdOut=io.airbyte.config.ResourceRequirements@237f6f41[cpuRequest=0.5,cpuLimit=1,memoryRequest=25Mi,memoryLimit=50Mi,additionalProperties={}]] and allowedHosts io.airbyte.config.AllowedHosts@b4b2793[hosts=[*.datadoghq.com, *.datadoghq.eu, *.sentry.io],additionalProperties={}]
2024-09-03 10:32:38 replication-orchestrator > Attempting to start pod = destination-weaviate-write-14-3-ikveb for airbyte/destination-weaviate:0.2.34 with resources ConnectorResourceRequirements[main=io.airbyte.config.ResourceRequirements@42b7157[cpuRequest=1,cpuLimit=3,memoryRequest=2Gi,memoryLimit=4Gi,additionalProperties={}], heartbeat=io.airbyte.config.ResourceRequirements@1d8a78d8[cpuRequest=0.05,cpuLimit=0.2,memoryRequest=25Mi,memoryLimit=50Mi,additionalProperties={}], stdErr=io.airbyte.config.ResourceRequirements@29b52f9d[cpuRequest=0.01,cpuLimit=0.5,memoryRequest=25Mi,memoryLimit=50Mi,additionalProperties={}], stdIn=io.airbyte.config.ResourceRequirements@5a7945ac[cpuRequest=0.5,cpuLimit=2,memoryRequest=25Mi,memoryLimit=50Mi,additionalProperties={}], stdOut=io.airbyte.config.ResourceRequirements@2d52b136[cpuRequest=0.01,cpuLimit=0.5,memoryRequest=25Mi,memoryLimit=50Mi,additionalProperties={}]] and allowedHosts io.airbyte.config.AllowedHosts@722cdc02[hosts=[z48ndi5vs0wjvwgzb7ga.c0.asia-southeast1.gcp.weaviate.cloud, api.openai.com, api.cohere.ai, aze-fc-openai-stg-create-01.openai.azure.com, *.datadoghq.com, *.datadoghq.eu, *.sentry.io],additionalProperties={}]
2024-09-03 10:32:38 replication-orchestrator > destination-weaviate-write-14-3-ikveb stdoutLocalPort = 9878
2024-09-03 10:32:38 replication-orchestrator > source-mongodb-v2-read-14-3-wxtaw stdoutLocalPort = 9877
2024-09-03 10:32:38 replication-orchestrator > destination-weaviate-write-14-3-ikveb stderrLocalPort = 9879
2024-09-03 10:32:38 replication-orchestrator > source-mongodb-v2-read-14-3-wxtaw stderrLocalPort = 9880
2024-09-03 10:32:38 replication-orchestrator > Creating stdout socket server...
2024-09-03 10:32:38 replication-orchestrator > Creating stdout socket server...
2024-09-03 10:32:38 replication-orchestrator > Creating stderr socket server...
2024-09-03 10:32:38 replication-orchestrator > Creating stderr socket server...
2024-09-03 10:32:38 replication-orchestrator > Creating pod destination-weaviate-write-14-3-ikveb...
2024-09-03 10:32:38 replication-orchestrator > Creating pod source-mongodb-v2-read-14-3-wxtaw...
2024-09-03 10:32:39 replication-orchestrator > Waiting for init container to be ready before copying files...
2024-09-03 10:32:39 replication-orchestrator > Waiting for init container to be ready before copying files...
2024-09-03 10:32:40 replication-orchestrator > Init container ready..
2024-09-03 10:32:40 replication-orchestrator > Copying files...
2024-09-03 10:32:40 replication-orchestrator > Uploading file: source_config.json
2024-09-03 10:32:40 replication-orchestrator > kubectl cp /tmp/92894c05-5def-491c-9404-f598ceaf9aa6/source_config.json airbyte-abctl/source-mongodb-v2-read-14-3-wxtaw:/config/source_config.json -c init --retries=3
2024-09-03 10:32:40 replication-orchestrator > Init container ready..
2024-09-03 10:32:40 replication-orchestrator > Copying files...
2024-09-03 10:32:40 replication-orchestrator > Uploading file: destination_config.json
2024-09-03 10:32:40 replication-orchestrator > kubectl cp /tmp/25a9c3ed-1d27-41f7-80dc-9638c4a3e099/destination_config.json airbyte-abctl/destination-weaviate-write-14-3-ikveb:/config/destination_config.json -c init --retries=3
2024-09-03 10:32:40 replication-orchestrator > Waiting for kubectl cp to complete
2024-09-03 10:32:40 replication-orchestrator > Waiting for kubectl cp to complete
2024-09-03 10:32:40 replication-orchestrator > kubectl cp complete, closing process
2024-09-03 10:32:40 replication-orchestrator > Uploading file: source_catalog.json
2024-09-03 10:32:40 replication-orchestrator > kubectl cp /tmp/9ebeb7ab-a2bd-4083-88b2-9d09aaec8467/source_catalog.json airbyte-abctl/source-mongodb-v2-read-14-3-wxtaw:/config/source_catalog.json -c init --retries=3
2024-09-03 10:32:40 replication-orchestrator > Waiting for kubectl cp to complete
2024-09-03 10:32:40 replication-orchestrator > kubectl cp complete, closing process
2024-09-03 10:32:40 replication-orchestrator > Uploading file: destination_catalog.json
2024-09-03 10:32:40 replication-orchestrator > kubectl cp /tmp/5eedfca0-0e93-465c-bab9-2f297c77aa6a/destination_catalog.json airbyte-abctl/destination-weaviate-write-14-3-ikveb:/config/destination_catalog.json -c init --retries=3
2024-09-03 10:32:40 replication-orchestrator > Waiting for kubectl cp to complete
2024-09-03 10:32:40 replication-orchestrator > kubectl cp complete, closing process
2024-09-03 10:32:40 replication-orchestrator > Uploading file: FINISHED_UPLOADING
2024-09-03 10:32:40 replication-orchestrator > kubectl cp /tmp/352c3a9d-2f94-4126-ba1e-50ec73cc0fad/FINISHED_UPLOADING airbyte-abctl/source-mongodb-v2-read-14-3-wxtaw:/config/FINISHED_UPLOADING -c init --retries=3
2024-09-03 10:32:40 replication-orchestrator > Waiting for kubectl cp to complete
2024-09-03 10:32:40 replication-orchestrator > kubectl cp complete, closing process
2024-09-03 10:32:40 replication-orchestrator > Uploading file: FINISHED_UPLOADING
2024-09-03 10:32:40 replication-orchestrator > kubectl cp /tmp/bc51bf8c-8c08-4ea8-8924-9bb80785e5ee/FINISHED_UPLOADING airbyte-abctl/destination-weaviate-write-14-3-ikveb:/config/FINISHED_UPLOADING -c init --retries=3
2024-09-03 10:32:40 replication-orchestrator > Waiting for kubectl cp to complete
2024-09-03 10:32:40 replication-orchestrator > kubectl cp complete, closing process
2024-09-03 10:32:40 replication-orchestrator > Waiting until pod is ready...
2024-09-03 10:32:40 replication-orchestrator > kubectl cp complete, closing process
2024-09-03 10:32:40 replication-orchestrator > Waiting until pod is ready...
2024-09-03 10:32:41 replication-orchestrator > Setting stdout...
2024-09-03 10:32:41 replication-orchestrator > Setting stderr...
2024-09-03 10:32:42 replication-orchestrator > Setting stdout...
2024-09-03 10:32:42 replication-orchestrator > Setting stderr...
2024-09-03 10:32:42 replication-orchestrator > Reading pod IP...
2024-09-03 10:32:42 replication-orchestrator > Pod IP: 10.244.0.116
2024-09-03 10:32:42 replication-orchestrator > Using null stdin output stream...
2024-09-03 10:32:42 replication-orchestrator > Reading messages from protocol version 0.2.0
2024-09-03 10:32:42 replication-orchestrator > Reading pod IP...
2024-09-03 10:32:42 replication-orchestrator > Pod IP: 10.244.0.117
2024-09-03 10:32:42 replication-orchestrator > Creating stdin socket...
2024-09-03 10:32:42 replication-orchestrator > Writing messages to protocol version 0.2.0
2024-09-03 10:32:42 replication-orchestrator > Reading messages from protocol version 0.2.0
2024-09-03 10:32:42 replication-orchestrator > Starting workload heartbeat
2024-09-03 10:32:42 replication-orchestrator > readFromSource: start
2024-09-03 10:32:42 replication-orchestrator > Starting source heartbeat check. Will check threshold of 10800 seconds, every 1 minutes.
2024-09-03 10:32:42 replication-orchestrator > processMessage: start
2024-09-03 10:32:42 replication-orchestrator > writeToDestination: start
2024-09-03 10:32:42 replication-orchestrator > readFromDestination: start
2024-09-03 10:32:42 source > INFO main i.a.i.s.m.MongoDbSource(main):52 starting source: class io.airbyte.integrations.source.mongodb.MongoDbSource
2024-09-03 10:32:43 source > INFO main i.a.c.i.b.IntegrationCliParser$Companion(parseOptions):144 integration args: {read=null, catalog=source_catalog.json, config=source_config.json}
2024-09-03 10:32:43 source > INFO main i.a.c.i.b.IntegrationRunner(runInternal):130 Running integration: io.airbyte.integrations.source.mongodb.MongoDbSource
2024-09-03 10:32:43 source > INFO main i.a.c.i.b.IntegrationRunner(runInternal):131 Command: READ
2024-09-03 10:32:43 source > INFO main i.a.c.i.b.IntegrationRunner(runInternal):132 Integration config: IntegrationConfig{command=READ, configPath='source_config.json', catalogPath='source_catalog.json', statePath='null'}
2024-09-03 10:32:43 source > WARN main c.n.s.JsonMetaSchema(newValidator):278 Unknown keyword groups - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2024-09-03 10:32:43 source > WARN main c.n.s.JsonMetaSchema(newValidator):278 Unknown keyword order - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2024-09-03 10:32:43 source > WARN main c.n.s.JsonMetaSchema(newValidator):278 Unknown keyword group - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2024-09-03 10:32:43 source > WARN main c.n.s.JsonMetaSchema(newValidator):278 Unknown keyword display_type - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2024-09-03 10:32:43 source > WARN main c.n.s.JsonMetaSchema(newValidator):278 Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2024-09-03 10:32:43 source > WARN main c.n.s.JsonMetaSchema(newValidator):278 Unknown keyword always_show - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2024-09-03 10:32:43 source > WARN main c.n.s.JsonMetaSchema(newValidator):278 Unknown keyword min - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2024-09-03 10:32:43 source > WARN main c.n.s.JsonMetaSchema(newValidator):278 Unknown keyword max - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2024-09-03 10:32:43 source > INFO cluster-ClusterId{value='66d6e5cbb0d9826afc003235', description='null'}-srv-cluster0.jwvtw.mongodb.net c.m.i.d.l.SLF4JLogger(info):71 Adding discovered server cluster0-shard-00-00.jwvtw.mongodb.net:27017 to client view of cluster
2024-09-03 10:32:43 source > INFO main c.m.i.d.l.SLF4JLogger(info):71 MongoClient with metadata {"driver": {"name": "mongo-java-driver|sync|Airbyte", "version": "4.11.0"}, "os": {"type": "Linux", "name": "Linux", "architecture": "amd64", "version": "5.15.153.1-microsoft-standard-WSL2"}, "platform": "Java/Amazon.com Inc./21.0.4+7-LTS"} created with settings MongoClientSettings{readPreference=ReadPreference{name=secondaryPreferred, hedgeOptions=null}, writeConcern=WriteConcern{w=null, wTimeout=null ms, journal=null}, retryWrites=true, retryReads=true, readConcern=ReadConcern{level=null}, credential=MongoCredential{mechanism=null, userName='READ_ONLY_USER', source='admin', password=<hidden>, mechanismProperties=<hidden>}, transportSettings=null, streamFactoryFactory=null, commandListeners=[], codecRegistry=ProvidersCodecRegistry{codecProviders=[ValueCodecProvider{}, BsonValueCodecProvider{}, DBRefCodecProvider{}, DBObjectCodecProvider{}, DocumentCodecProvider{}, CollectionCodecProvider{}, IterableCodecProvider{}, MapCodecProvider{}, GeoJsonCodecProvider{}, GridFSFileCodecProvider{}, Jsr310CodecProvider{}, JsonObjectCodecProvider{}, BsonCodecProvider{}, EnumCodecProvider{}, com.mongodb.client.model.mql.ExpressionCodecProvider@5a9800f8, com.mongodb.Jep395RecordCodecProvider@143d9a93, com.mongodb.KotlinCodecProvider@40226788]}, loggerSettings=LoggerSettings{maxDocumentLength=1000}, clusterSettings={hosts=[127.0.0.1:27017], srvHost=cluster0.jwvtw.mongodb.net, srvServiceName=mongodb, mode=MULTIPLE, requiredClusterType=REPLICA_SET, requiredReplicaSetName='atlas-3fnexm-shard-0', serverSelector='null', clusterListeners='[]', serverSelectionTimeout='30000 ms', localThreshold='15 ms'}, socketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=0, receiveBufferSize=0, proxySettings=ProxySettings{host=null, port=null, username=null, password=null}}, heartbeatSocketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=10000, receiveBufferSize=0, proxySettings=ProxySettings{host=null, port=null, username=null, password=null}}, connectionPoolSettings=ConnectionPoolSettings{maxSize=100, minSize=0, maxWaitTimeMS=120000, maxConnectionLifeTimeMS=0, maxConnectionIdleTimeMS=0, maintenanceInitialDelayMS=0, maintenanceFrequencyMS=60000, connectionPoolListeners=[], maxConnecting=2}, serverSettings=ServerSettings{heartbeatFrequencyMS=10000, minHeartbeatFrequencyMS=500, serverListeners='[]', serverMonitorListeners='[]'}, sslSettings=SslSettings{enabled=true, invalidHostNameAllowed=false, context=null}, applicationName='null', compressorList=[], uuidRepresentation=UNSPECIFIED, serverApi=null, autoEncryptionSettings=null, dnsClient=null, inetAddressResolver=null, contextProvider=null}
2024-09-03 10:32:43 source > INFO main i.a.i.s.m.MongoDbSource(read):148 There are 1 Full refresh streams
2024-09-03 10:32:43 source > INFO cluster-ClusterId{value='66d6e5cbb0d9826afc003235', description='null'}-srv-cluster0.jwvtw.mongodb.net c.m.i.d.l.SLF4JLogger(info):71 Adding discovered server cluster0-shard-00-01.jwvtw.mongodb.net:27017 to client view of cluster
2024-09-03 10:32:43 source > INFO cluster-ClusterId{value='66d6e5cbb0d9826afc003235', description='null'}-srv-cluster0.jwvtw.mongodb.net c.m.i.d.l.SLF4JLogger(info):71 Adding discovered server cluster0-shard-00-02.jwvtw.mongodb.net:27017 to client view of cluster
2024-09-03 10:32:43 source > INFO main c.m.i.d.l.SLF4JLogger(info):71 No server chosen by ReadPreferenceServerSelector{readPreference=ReadPreference{name=secondaryPreferred, hedgeOptions=null}} from cluster description ClusterDescription{type=REPLICA_SET, connectionMode=MULTIPLE, serverDescriptions=[ServerDescription{address=cluster0-shard-00-00.jwvtw.mongodb.net:27017, type=UNKNOWN, state=CONNECTING}, ServerDescription{address=cluster0-shard-00-01.jwvtw.mongodb.net:27017, type=UNKNOWN, state=CONNECTING}, ServerDescription{address=cluster0-shard-00-02.jwvtw.mongodb.net:27017, type=UNKNOWN, state=CONNECTING}]}. Waiting for 30000 ms before timing out
2024-09-03 10:32:44 source > INFO cluster-ClusterId{value='66d6e5cbb0d9826afc003235', description='null'}-cluster0-shard-00-02.jwvtw.mongodb.net:27017 c.m.i.d.l.SLF4JLogger(info):71 Monitor thread successfully connected to server with description ServerDescription{address=cluster0-shard-00-02.jwvtw.mongodb.net:27017, type=REPLICA_SET_SECONDARY, state=CONNECTED, ok=true, minWireVersion=0, maxWireVersion=21, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=361853846, setName='atlas-3fnexm-shard-0', canonicalAddress=cluster0-shard-00-02.jwvtw.mongodb.net:27017, hosts=[cluster0-shard-00-00.jwvtw.mongodb.net:27017, cluster0-shard-00-01.jwvtw.mongodb.net:27017, cluster0-shard-00-02.jwvtw.mongodb.net:27017], passives=[], arbiters=[], primary='cluster0-shard-00-01.jwvtw.mongodb.net:27017', tagSet=TagSet{[Tag{name='availabilityZone', value='aps1-az2'}, Tag{name='diskState', value='READY'}, Tag{name='nodeType', value='ELECTABLE'}, Tag{name='provider', value='AWS'}, Tag{name='region', value='AP_SOUTH_1'}, Tag{name='workloadType', value='OPERATIONAL'}]}, electionId=null, setVersion=92, topologyVersion=TopologyVersion{processId=66cc3f7e08f6df3938bbe157, counter=5}, lastWriteDate=Tue Sep 03 10:32:43 UTC 2024, lastUpdateTimeNanos=10400808632683}
2024-09-03 10:32:44 source > INFO cluster-ClusterId{value='66d6e5cbb0d9826afc003235', description='null'}-cluster0-shard-00-00.jwvtw.mongodb.net:27017 c.m.i.d.l.SLF4JLogger(info):71 Monitor thread successfully connected to server with description ServerDescription{address=cluster0-shard-00-00.jwvtw.mongodb.net:27017, type=REPLICA_SET_SECONDARY, state=CONNECTED, ok=true, minWireVersion=0, maxWireVersion=21, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=346780710, setName='atlas-3fnexm-shard-0', canonicalAddress=cluster0-shard-00-00.jwvtw.mongodb.net:27017, hosts=[cluster0-shard-00-00.jwvtw.mongodb.net:27017, cluster0-shard-00-01.jwvtw.mongodb.net:27017, cluster0-shard-00-02.jwvtw.mongodb.net:27017], passives=[], arbiters=[], primary='cluster0-shard-00-01.jwvtw.mongodb.net:27017', tagSet=TagSet{[Tag{name='availabilityZone', value='aps1-az1'}, Tag{name='diskState', value='READY'}, Tag{name='nodeType', value='ELECTABLE'}, Tag{name='provider', value='AWS'}, Tag{name='region', value='AP_SOUTH_1'}, Tag{name='workloadType', value='OPERATIONAL'}]}, electionId=null, setVersion=92, topologyVersion=TopologyVersion{processId=66cc3e35804a86ace670d69e, counter=5}, lastWriteDate=Tue Sep 03 10:32:43 UTC 2024, lastUpdateTimeNanos=10400808667511}
2024-09-03 10:32:44 source > INFO cluster-ClusterId{value='66d6e5cbb0d9826afc003235', description='null'}-cluster0-shard-00-01.jwvtw.mongodb.net:27017 c.m.i.d.l.SLF4JLogger(info):71 Monitor thread successfully connected to server with description ServerDescription{address=cluster0-shard-00-01.jwvtw.mongodb.net:27017, type=REPLICA_SET_PRIMARY, state=CONNECTED, ok=true, minWireVersion=0, maxWireVersion=21, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=380627877, setName='atlas-3fnexm-shard-0', canonicalAddress=cluster0-shard-00-01.jwvtw.mongodb.net:27017, hosts=[cluster0-shard-00-00.jwvtw.mongodb.net:27017, cluster0-shard-00-01.jwvtw.mongodb.net:27017, cluster0-shard-00-02.jwvtw.mongodb.net:27017], passives=[], arbiters=[], primary='cluster0-shard-00-01.jwvtw.mongodb.net:27017', tagSet=TagSet{[Tag{name='availabilityZone', value='aps1-az3'}, Tag{name='diskState', value='READY'}, Tag{name='nodeType', value='ELECTABLE'}, Tag{name='provider', value='AWS'}, Tag{name='region', value='AP_SOUTH_1'}, Tag{name='workloadType', value='OPERATIONAL'}]}, electionId=7fffffff000000000000005f, setVersion=92, topologyVersion=TopologyVersion{processId=66cc3d11432d320fa42dc774, counter=9}, lastWriteDate=Tue Sep 03 10:32:43 UTC 2024, lastUpdateTimeNanos=10400842565877}
2024-09-03 10:32:44 source > INFO cluster-ClusterId{value='66d6e5cbb0d9826afc003235', description='null'}-cluster0-shard-00-01.jwvtw.mongodb.net:27017 c.m.i.d.l.SLF4JLogger(info):71 Discovered replica set primary cluster0-shard-00-01.jwvtw.mongodb.net:27017 with max election id 7fffffff000000000000005f and max set version 92
2024-09-03 10:32:44 source > INFO main i.a.i.s.m.MongoUtil(getChunkSizeForCollection):239 Chunk size determined for: smart_solve_db.smart_solve, to be 88098 rows
2024-09-03 10:32:45 replication-orchestrator > Stream status TRACE received of status: STARTED for stream smart_solve_db:smart_solve
2024-09-03 10:32:45 replication-orchestrator > Sending update for smart_solve_db:smart_solve - null -> RUNNING
2024-09-03 10:32:45 replication-orchestrator > Stream Status Update Received: smart_solve_db:smart_solve - RUNNING
2024-09-03 10:32:45 replication-orchestrator > Creating status: smart_solve_db:smart_solve - RUNNING
2024-09-03 10:32:45 destination > Begin writing to the destination...
2024-09-03 10:32:45 source > INFO main i.a.i.s.m.MongoDbInitialLoadRecordIterator(computeNext):89 Finishing subquery number : 0, processing at id : f67735e7e7c54032b8c55244022986bc
2024-09-03 10:32:45 source > INFO main i.a.c.i.s.r.s.SourceStateIterator(computeNext):84 sending final state message, with count per stream: {smart_solve_db_smart_solve=10} 
2024-09-03 10:32:45 source > INFO main i.a.c.i.b.IntegrationRunner(runInternal):224 Completed integration: io.airbyte.integrations.source.mongodb.MongoDbSource
2024-09-03 10:32:45 source > INFO main i.a.i.s.m.MongoDbSource(main):54 completed source: class io.airbyte.integrations.source.mongodb.MongoDbSource
2024-09-03 10:32:45 destination > /usr/local/lib/python3.10/site-packages/langchain_core/_api/deprecation.py:117: LangChainDeprecationWarning: The class `langchain_community.embeddings.openai.OpenAIEmbeddings` was deprecated in langchain-community 0.0.9 and will be removed in 0.2.0. An updated version of the class exists in the langchain-openai package and should be used instead. To use it run `pip install -U langchain-openai` and import as `from langchain_openai import OpenAIEmbeddings`.
2024-09-03 10:32:45 destination >   warn_deprecated(
2024-09-03 10:32:45 replication-orchestrator > Stream status TRACE received of status: COMPLETE for stream smart_solve_db:smart_solve
2024-09-03 10:32:46 replication-orchestrator > (pod: airbyte-abctl / source-mongodb-v2-read-14-3-wxtaw) - Closed all resources for pod
2024-09-03 10:32:46 replication-orchestrator > Total records read: 13 (120 KB)
2024-09-03 10:32:46 replication-orchestrator > Schema validation was performed to a max of 10 records with errors per stream.
2024-09-03 10:32:46 replication-orchestrator > Schema validation errors found for stream smart_solve_db_smart_solve. Error messages: [$.Deleted: boolean found, string expected, $.NeedsUpdate: boolean found, string expected, $.Solution: null found, string expected, $.Issue: null found, string expected, $.PartnerNotes: string found, null expected, $.IsIssueDoc: boolean found, string expected, $.NeedsNLPUpdate: boolean found, string expected, $.InternalNotes: null found, string expected, $.NeedsReview: boolean found, string expected, $.Answer: string found, null expected, $.Procedure: string found, null expected, $.Description: string found, null expected, $.Cause: null found, string expected]
2024-09-03 10:32:46 replication-orchestrator > readFromSource: done. (source.isFinished:true, fromSource.isClosed:false)
2024-09-03 10:32:46 replication-orchestrator > thread status... heartbeat thread: false , replication thread: true
2024-09-03 10:32:50 replication-orchestrator > processMessage: done. (fromSource.isDone:true, forDest.isClosed:false)
2024-09-03 10:32:50 replication-orchestrator > writeToDestination: done. (forDest.isDone:true, isDestRunning:true)
2024-09-03 10:32:50 replication-orchestrator > thread status... timeout thread: false , replication thread: true
2024-09-03 10:32:53 destination > /usr/local/lib/python3.10/site-packages/weaviate/warnings.py:121: DeprecationWarning: Dep005: You are using weaviate-client version 3.25.2. The latest version is 4.7.1.
2024-09-03 10:32:53 destination >             Please consider upgrading to the latest version. See https://weaviate.io/developers/weaviate/client-libraries/python for details.
2024-09-03 10:32:53 destination >   warnings.warn(
2024-09-03 10:33:05 destination > /usr/local/lib/python3.10/site-packages/weaviate/warnings.py:80: DeprecationWarning: Dep002: You are batching manually. This means you are NOT using the client's built-in
2024-09-03 10:33:05 destination >             multi-threading. Setting `batch_size` in `client.batch.configure()`  to an int value will enabled automatic
2024-09-03 10:33:05 destination >             batching. See:
2024-09-03 10:33:05 destination >             https://weaviate.io/developers/weaviate/current/restful-api-references/batch.html#example-request-1
2024-09-03 10:33:05 destination >   warnings.warn(
2024-09-03 10:33:26 replication-orchestrator > Destination complete for GLOBAL
2024-09-03 10:33:26 replication-orchestrator > Stream Status Update Received: smart_solve_db:smart_solve - COMPLETE
2024-09-03 10:33:26 replication-orchestrator > Updating status: smart_solve_db:smart_solve - COMPLETE
2024-09-03 10:33:26 replication-orchestrator > readFromDestination: exception caught
java.lang.NullPointerException: Cannot invoke "io.airbyte.protocol.models.AirbyteGlobalState.getStreamStates()" because the return value of "io.airbyte.protocol.models.AirbyteStateMessage.getGlobal()" is null
    at io.airbyte.workers.internal.bookkeeping.ParallelStreamStatsTracker.updateDestinationStateStats(ParallelStreamStatsTracker.kt:130) ~[io.airbyte-airbyte-commons-worker-0.64.1.jar:?]
    at io.airbyte.workers.internal.syncpersistence.SyncPersistenceImpl.updateDestinationStateStats(SyncPersistence.kt:322) ~[io.airbyte-airbyte-commons-worker-0.64.1.jar:?]
    at io.airbyte.workers.internal.bookkeeping.AirbyteMessageTracker.acceptFromDestination(AirbyteMessageTracker.kt:65) ~[io.airbyte-airbyte-commons-worker-0.64.1.jar:?]
    at io.airbyte.workers.general.ReplicationWorkerHelper.internalProcessMessageFromDestination(ReplicationWorkerHelper.kt:413) ~[io.airbyte-airbyte-commons-worker-0.64.1.jar:?]
    at io.airbyte.workers.general.ReplicationWorkerHelper.processMessageFromDestination(ReplicationWorkerHelper.kt:302) ~[io.airbyte-airbyte-commons-worker-0.64.1.jar:?]
    at io.airbyte.workers.general.BufferedReplicationWorker.readFromDestination(BufferedReplicationWorker.java:478) ~[io.airbyte-airbyte-commons-worker-0.64.1.jar:?]
    at io.airbyte.workers.general.BufferedReplicationWorker.lambda$runAsync$2(BufferedReplicationWorker.java:219) ~[io.airbyte-airbyte-commons-worker-0.64.1.jar:?]
    at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804) ~[?:?]
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
    at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
2024-09-03 10:33:26 replication-orchestrator > readFromDestination: done. (writeToDestFailed:false, dest.isFinished:false)
2024-09-03 10:33:27 replication-orchestrator > (pod: airbyte-abctl / destination-weaviate-write-14-3-ikveb) - Closed all resources for pod
2024-09-03 10:33:27 replication-orchestrator > thread status... timeout thread: false , replication thread: true
2024-09-03 10:33:27 replication-orchestrator > Closing StateCheckSumCountEventHandler
2024-09-03 10:33:27 replication-orchestrator > sync summary: {
  "status" : "failed",
  "recordsSynced" : 0,
  "bytesSynced" : 0,
  "startTime" : 1725359558265,
  "endTime" : 1725359607588,
  "totalStats" : {
    "bytesCommitted" : 0,
    "bytesEmitted" : 123718,
    "destinationStateMessagesEmitted" : 0,
    "destinationWriteEndTime" : 0,
    "destinationWriteStartTime" : 1725359558321,
    "meanSecondsBeforeSourceStateMessageEmitted" : 0,
    "maxSecondsBeforeSourceStateMessageEmitted" : 0,
    "maxSecondsBetweenStateMessageEmittedandCommitted" : 0,
    "meanSecondsBetweenStateMessageEmittedandCommitted" : 0,
    "recordsEmitted" : 10,
    "recordsCommitted" : 0,
    "replicationEndTime" : 1725359607579,
    "replicationStartTime" : 1725359558265,
    "sourceReadEndTime" : 1725359566383,
    "sourceReadStartTime" : 1725359558324,
    "sourceStateMessagesEmitted" : 1
  },
  "streamStats" : [ {
    "streamName" : "smart_solve",
    "streamNamespace" : "smart_solve_db",
    "stats" : {
      "bytesCommitted" : 0,
      "bytesEmitted" : 123718,
      "recordsEmitted" : 10,
      "recordsCommitted" : 0
    }
  } ],
  "performanceMetrics" : {
    "processFromSource" : {
      "elapsedTimeInNanos" : 173313903,
      "executionCount" : 13,
      "avgExecTimeInNanos" : 1.3331838692307692E7
    },
    "readFromSource" : {
      "elapsedTimeInNanos" : 3863459900,
      "executionCount" : 247068,
      "avgExecTimeInNanos" : 15637.233069438373
    },
    "processFromDest" : {
      "elapsedTimeInNanos" : 187349612,
      "executionCount" : 1,
      "avgExecTimeInNanos" : 1.87349612E8
    },
    "writeToDest" : {
      "elapsedTimeInNanos" : 212949727,
      "executionCount" : 11,
      "avgExecTimeInNanos" : 1.935906609090909E7
    },
    "readFromDest" : {
      "elapsedTimeInNanos" : 43999425921,
      "executionCount" : 2,
      "avgExecTimeInNanos" : 2.19997129605E10
    }
  }
}
2024-09-03 10:33:27 replication-orchestrator > failures: [ {
  "failureOrigin" : "replication",
  "internalMessage" : "Cannot invoke \"io.airbyte.protocol.models.AirbyteGlobalState.getStreamStates()\" because the return value of \"io.airbyte.protocol.models.AirbyteStateMessage.getGlobal()\" is null",
  "externalMessage" : "Something went wrong during replication",
  "metadata" : {
    "attemptNumber" : 3,
    "jobId" : 14
  },
  "stacktrace" : "java.lang.NullPointerException: Cannot invoke \"io.airbyte.protocol.models.AirbyteGlobalState.getStreamStates()\" because the return value of \"io.airbyte.protocol.models.AirbyteStateMessage.getGlobal()\" is null\n\tat io.airbyte.workers.internal.bookkeeping.ParallelStreamStatsTracker.updateDestinationStateStats(ParallelStreamStatsTracker.kt:130)\n\tat io.airbyte.workers.internal.syncpersistence.SyncPersistenceImpl.updateDestinationStateStats(SyncPersistence.kt:322)\n\tat io.airbyte.workers.internal.bookkeeping.AirbyteMessageTracker.acceptFromDestination(AirbyteMessageTracker.kt:65)\n\tat io.airbyte.workers.general.ReplicationWorkerHelper.internalProcessMessageFromDestination(ReplicationWorkerHelper.kt:413)\n\tat io.airbyte.workers.general.ReplicationWorkerHelper.processMessageFromDestination(ReplicationWorkerHelper.kt:302)\n\tat io.airbyte.workers.general.BufferedReplicationWorker.readFromDestination(BufferedReplicationWorker.java:478)\n\tat io.airbyte.workers.general.BufferedReplicationWorker.lambda$runAsync$2(BufferedReplicationWorker.java:219)\n\tat java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)\n\tat java.base/java.lang.Thread.run(Thread.java:1583)\n",
  "timestamp" : 1725359606566
} ]
2024-09-03 10:33:27 replication-orchestrator > 
2024-09-03 10:33:27 replication-orchestrator > ----- END REPLICATION -----
2024-09-03 10:33:27 replication-orchestrator > 
2024-09-03 10:33:28 replication-orchestrator > Returning output...
2024-09-03 10:33:57 platform > Workload 4263d587-4ae4-4995-b0d1-7e970c9b1322_14_3_sync has returned a terminal status of failure.  Fetching output...
2024-09-03 10:33:57 platform > Replication output for workload 4263d587-4ae4-4995-b0d1-7e970c9b1322_14_3_sync : io.airbyte.config.ReplicationOutput@2489a7cf[replicationAttemptSummary=io.airbyte.config.ReplicationAttemptSummary@5622e63f[status=failed,recordsSynced=0,bytesSynced=0,startTime=1725359558265,endTime=1725359607588,totalStats=io.airbyte.config.SyncStats@6f0c79bf[bytesCommitted=0,bytesEmitted=123718,destinationStateMessagesEmitted=0,destinationWriteEndTime=0,destinationWriteStartTime=1725359558321,estimatedBytes=<null>,estimatedRecords=<null>,meanSecondsBeforeSourceStateMessageEmitted=0,maxSecondsBeforeSourceStateMessageEmitted=0,maxSecondsBetweenStateMessageEmittedandCommitted=0,meanSecondsBetweenStateMessageEmittedandCommitted=0,recordsEmitted=10,recordsCommitted=0,replicationEndTime=1725359607579,replicationStartTime=1725359558265,sourceReadEndTime=1725359566383,sourceReadStartTime=1725359558324,sourceStateMessagesEmitted=1,discoverSchemaEndTime=<null>,discoverSchemaStartTime=<null>,additionalProperties={}],streamStats=[io.airbyte.config.StreamSyncStats@29eebd02[streamName=smart_solve,streamNamespace=smart_solve_db,stats=io.airbyte.config.SyncStats@40b0c02f[bytesCommitted=0,bytesEmitted=123718,destinationStateMessagesEmitted=<null>,destinationWriteEndTime=<null>,destinationWriteStartTime=<null>,estimatedBytes=<null>,estimatedRecords=<null>,meanSecondsBeforeSourceStateMessageEmitted=<null>,maxSecondsBeforeSourceStateMessageEmitted=<null>,maxSecondsBetweenStateMessageEmittedandCommitted=<null>,meanSecondsBetweenStateMessageEmittedandCommitted=<null>,recordsEmitted=10,recordsCommitted=0,replicationEndTime=<null>,replicationStartTime=<null>,sourceReadEndTime=<null>,sourceReadStartTime=<null>,sourceStateMessagesEmitted=<null>,discoverSchemaEndTime=<null>,discoverSchemaStartTime=<null>,additionalProperties={}],wasBackfilled=<null>,wasResumed=<null>,additionalProperties={}]],performanceMetrics=io.airbyte.config.PerformanceMetrics@162c518[additionalProperties={processFromSource={elapsedTimeInNanos=173313903, executionCount=13, avgExecTimeInNanos=1.3331838692307692E7}, readFromSource={elapsedTimeInNanos=3863459900, executionCount=247068, avgExecTimeInNanos=15637.233069438373}, processFromDest={elapsedTimeInNanos=187349612, executionCount=1, avgExecTimeInNanos=1.87349612E8}, writeToDest={elapsedTimeInNanos=212949727, executionCount=11, avgExecTimeInNanos=1.935906609090909E7}, readFromDest={elapsedTimeInNanos=43999425921, executionCount=2, avgExecTimeInNanos=2.19997129605E10}}],additionalProperties={}],state=<null>,outputCatalog=ConfiguredAirbyteCatalog(streams=[ConfiguredAirbyteStream(stream=AirbyteStream(name=smart_solve, jsonSchema={"type":"object","properties":{"URI":{"type":"string"},"_id":{"type":"string"},"Tags":{"type":"object"},"Text":{"type":"string"},"Cause":{"type":"string"},"Issue":{"type":"string"},"L1t6m":{"type":"number"},"L2t6m":{"type":"number"},"Title":{"type":"string"},"Views":{"type":"number"},"Answer":{"type":"null"},"Created":{"type":"string"},"Deleted":{"type":"string"},"Feature":{"type":"string"},"L1cases":{"type":"array"},"L2cases":{"type":"array"},"Category":{"type":"string"},"Grouping":{"type":"string"},"Modified":{"type":"string"},"Solution":{"type":"string"},"AppliesTo":{"type":"string"},"Procedure":{"type":"null"},"Unique_Id":{"type":"string"},"CaseDriver":{"type":"string"},"CaseMonths":{"type":"array"},"IsIssueDoc":{"type":"string"},"Resolution":{"type":"string"},"Description":{"type":"null"},"L1CaseCount":{"type":"number"},"L2CaseCount":{"type":"number"},"NeedsReview":{"type":"string"},"NeedsUpdate":{"type":"string"},"InboundLinks":{"type":"array"},"PartnerNotes":{"type":"null"},"InternalNotes":{"type":"string"},"OutboundLinks":{"type":"array"},"ProductBranch":{"type":"string"},"NeedsNLPUpdate":{"type":"string"},"_ab_cdc_cursor":{"type":"number","airbyte_type":"integer"},"SolutionSentiment":{"type":"null"},"_ab_cdc_deleted_at":{"type":"string"},"_ab_cdc_updated_at":{"type":"string"},"AdditionalInformation":{"type":"string"}},"$schema":"http://json-schema.org/draft-07/schema#"}, supportedSyncModes=[full_refresh, incremental], sourceDefinedCursor=true, defaultCursorField=[_ab_cdc_cursor], sourceDefinedPrimaryKey=[[_id]], namespace=null, isResumable=true), syncMode=full_refresh, destinationSyncMode=overwrite, cursorField=[_ab_cdc_cursor], primaryKey=[[_id]], generationId=null, minimumGenerationId=null, syncId=null, fields=null, mappers=[])]),failures=[io.airbyte.config.FailureReason@daf950a[failureOrigin=replication,failureType=<null>,internalMessage=Cannot invoke "io.airbyte.protocol.models.AirbyteGlobalState.getStreamStates()" because the return value of "io.airbyte.protocol.models.AirbyteStateMessage.getGlobal()" is null,externalMessage=Something went wrong during replication,metadata=io.airbyte.config.Metadata@51752f10[additionalProperties={attemptNumber=3, jobId=14}],stacktrace=java.lang.NullPointerException: Cannot invoke "io.airbyte.protocol.models.AirbyteGlobalState.getStreamStates()" because the return value of "io.airbyte.protocol.models.AirbyteStateMessage.getGlobal()" is null
    at io.airbyte.workers.internal.bookkeeping.ParallelStreamStatsTracker.updateDestinationStateStats(ParallelStreamStatsTracker.kt:130)
    at io.airbyte.workers.internal.syncpersistence.SyncPersistenceImpl.updateDestinationStateStats(SyncPersistence.kt:322)
    at io.airbyte.workers.internal.bookkeeping.AirbyteMessageTracker.acceptFromDestination(AirbyteMessageTracker.kt:65)
    at io.airbyte.workers.general.ReplicationWorkerHelper.internalProcessMessageFromDestination(ReplicationWorkerHelper.kt:413)
    at io.airbyte.workers.general.ReplicationWorkerHelper.processMessageFromDestination(ReplicationWorkerHelper.kt:302)
    at io.airbyte.workers.general.BufferedReplicationWorker.readFromDestination(BufferedReplicationWorker.java:478)
    at io.airbyte.workers.general.BufferedReplicationWorker.lambda$runAsync$2(BufferedReplicationWorker.java:219)
    at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
    at java.base/java.lang.Thread.run(Thread.java:1583)
,retryable=<null>,timestamp=1725359606566,streamDescriptor=<null>,additionalProperties={}]],additionalProperties={}]
2024-09-03 10:33:57 platform > No post-replication operation(s) to perform.
2024-09-03 10:33:57 platform > 
2024-09-03 10:33:57 platform > ----- START POST REPLICATION OPERATIONS -----
2024-09-03 10:33:57 platform > 
2024-09-03 10:33:57 platform > 
2024-09-03 10:33:57 platform > ----- END POST REPLICATION OPERATIONS -----
2024-09-03 10:33:57 platform > 
2024-09-03 10:33:57 platform > Retry State: RetryManager(completeFailureBackoffPolicy=BackoffPolicy(minInterval=PT10S, maxInterval=PT30M, base=3), partialFailureBackoffPolicy=null, successiveCompleteFailureLimit=5, totalCompleteFailureLimit=10, successivePartialFailureLimit=1000, totalPartialFailureLimit=20, successiveCompleteFailures=4, totalCompleteFailures=4, successivePartialFailures=0, totalPartialFailures=0)
 Backoff before next attempt: 4 minutes 30 seconds

Contribute

Yatindra29 commented 2 months ago

Any updates on this?

N-liraj-khanna commented 2 months ago

It havent been fixed yet, im facing the same issue

Hashcode-Ankit commented 1 month ago

I am also facing same issue, in my case record count is also not matching which got loaded in s3

I am running latest airbyte cloud, CDC sync of Mongodb. Resume Token is also valid. How I checked wrong count:

One more thing to add here: Record size was 2 MB.

teetangh commented 3 weeks ago

i am also facing the same isssue

replication-orchestrator > readFromDestination: exception caught
java.lang.NullPointerException: Cannot invoke "io.airbyte.protocol.models.AirbyteGlobalState.getStreamStates()" because the return value of "io.airbyte.protocol.models.AirbyteStateMessage.getGlobal()" is null
    at io.airbyte.workers.internal.bookkeeping.ParallelStreamStatsTracker.updateDestinationStateStats(ParallelStreamStatsTracker.kt:130) ~[io.airbyte-airbyte-commons-worker-1.1.0.jar:?]
    at io.airbyte.workers.internal.syncpersistence.SyncPersistenceImpl.updateDestinationStateStats(SyncPersistence.kt:322) ~[io.airbyte-airbyte-commons-worker-1.1.0.jar:?]
    at io.airbyte.workers.internal.bookkeeping.AirbyteMessageTracker.acceptFromDestination(AirbyteMessageTracker.kt:65) ~[io.airbyte-airbyte-commons-worker-1.1.0.jar:?]
    at io.airbyte.workers.general.ReplicationWorkerHelper.internalProcessMessageFromDestination(ReplicationWorkerHelper.kt:443) ~[io.airbyte-airbyte-commons-worker-1.1.0.jar:?]
    at io.airbyte.workers.general.ReplicationWorkerHelper.processMessageFromDestination(ReplicationWorkerHelper.kt:317) ~[io.airbyte-airbyte-commons-worker-1.1.0.jar:?]
    at io.airbyte.workers.general.BufferedReplicationWorker.readFromDestination(BufferedReplicationWorker.java:488) ~[io.airbyte-airbyte-commons-worker-1.1.0.jar:?]
    at io.airbyte.workers.general.BufferedReplicationWorker.lambda$runAsync$2(BufferedReplicationWorker.java:215) ~[io.airbyte-airbyte-commons-worker-1.1.0.jar:?]
    at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804) ~[?:?]
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
    at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
2024-10-28 11:40:50 replication-orchestrator > readFromDestination: done. (writeToDestFailed:false, dest.isFinished:false)
2024-10-28 11:40:50 replication-orchestrator > writeToDestination: exception caught
java.lang.IllegalStateException: No exit code found.
    at io.airbyte.workers.internal.ContainerIOHandle.getExitCode(ContainerIOHandle.kt:104) ~[io.airbyte-airbyte-commons-worker-1.1.0.jar:?]
    at io.airbyte.workers.internal.LocalContainerAirbyteSource.getExitValue(LocalContainerAirbyteSource.kt:90) ~[io.airbyte-airbyte-commons-worker-1.1.0.jar:?]
    at io.airbyte.workers.general.BufferedReplicationWorker.writeToDestination(BufferedReplicationWorker.java:440) ~[io.airbyte-airbyte-commons-worker-1.1.0.jar:?]
    at io.airbyte.workers.general.BufferedReplicationWorker.lambda$runAsyncWithTimeout$5(BufferedReplicationWorker.java:243) ~[io.airbyte-airbyte-commons-worker-1.1.0.jar:?]
    at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804) ~[?:?]
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
    at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]