search

airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
15.55k stars 4.01k forks source link

[destination-postgres] sync hangs indefinitely with no error #38534

Closed trentwaskey closed 4 months ago

trentwaskey commented 4 months ago

Connector Name

linkedin-ads

Connector Version

0.1.1

What step the error happened?

During the sync

Relevant information

Steps to reproduce:

  1. Connect LinkedIn Ads source
  2. Run initial sync
  3. Sync hangs, shows 0bytes and 0 rows created

Relevant log output

=== First failed initial sync ===
2024-05-21 15:19:10 INFO () WorkerRun(call):49 - Executing worker wrapper. Airbyte version: 0.30.25-alpha
2024-05-21 15:19:11 INFO () TemporalAttemptExecution(get):94 - Executing worker wrapper. Airbyte version: 0.30.25-alpha
2024-05-21 15:19:11 WARN () Databases(createPostgresDatabaseWithRetry):38 - Waiting for database to become available...
2024-05-21 15:19:11 INFO () JobsDatabaseInstance(lambda$static$2):25 - Testing if jobs database is ready...
2024-05-21 15:19:11 INFO () Databases(createPostgresDatabaseWithRetry):55 - Database available!
2024-05-21 15:19:11 WARN () JsonMetaSchema(newValidator):338 - Unknown keyword example - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2024-05-21 15:19:11 INFO () DefaultReplicationWorker(run):82 - start sync worker. job id: 1 attempt id: 0
2024-05-21 15:19:11 INFO () DefaultReplicationWorker(run):91 - configured sync modes: {null.accounts=full_refresh - append, null.ad_creative_analytics=full_refresh - append, null.campaign_groups=full_refresh - append, null.creatives=full_refresh - append, null.ad_direct_sponsored_contents=full_refresh - append, null.campaigns=full_refresh - append, null.ad_campaign_analytics=full_refresh - append}
2024-05-21 15:19:11 INFO () DefaultAirbyteDestination(start):65 - Running destination...
2024-05-21 15:19:11 INFO () LineGobbler(voidCall):82 - Checking if airbyte/destination-postgres:0.3.11 exists...
2024-05-21 15:19:11 INFO () DockerProcessFactory(create):127 - Preparing command: docker run --rm --init -i -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -w /data/1/0 --network host --log-driver none airbyte/destination-postgres:0.3.11 write --config destination_config.json --catalog destination_catalog.json
2024-05-21 15:19:11 INFO () LineGobbler(voidCall):82 - Checking if airbyte/source-linkedin-ads:0.1.1 exists...
2024-05-21 15:19:12 INFO () DockerProcessFactory(create):127 - Preparing command: docker run --rm --init -i -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -w /data/1/0 --network host --log-driver none airbyte/source-linkedin-ads:0.1.1 read --config source_config.json --catalog source_catalog.json
2024-05-21 15:19:12 INFO () DefaultReplicationWorker(lambda$getDestinationOutputRunnable$3):226 - Destination output thread started.
2024-05-21 15:19:12 INFO () DefaultReplicationWorker(run):119 - Waiting for source thread to join.
2024-05-21 15:19:12 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):190 - Replication thread started.
2024-05-21 15:19:16 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 15:19:16 INFO i.a.i.d.p.PostgresDestination(main):85 - {} - starting destination: class io.airbyte.integrations.destination.postgres.PostgresDestination
source - 2024-05-21 15:19:17 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 15:19:17 INFO i.a.i.b.IntegrationRunner(run):96 - {} - Running integration: io.airbyte.integrations.base.ssh.SshWrappedDestination
source - 2024-05-21 15:19:17 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 15:19:17 INFO i.a.i.b.IntegrationCliParser(parseOptions):135 - {} - integration args: {catalog=destination_catalog.json, write=null, config=destination_config.json}
source - 2024-05-21 15:19:17 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 15:19:17 INFO i.a.i.b.IntegrationRunner(run):100 - {} - Command: WRITE
source - 2024-05-21 15:19:17 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 15:19:17 INFO i.a.i.b.IntegrationRunner(run):101 - {} - Integration config: IntegrationConfig{command=WRITE, configPath='destination_config.json', catalogPath='destination_catalog.json', statePath='null'}
source - 2024-05-21 15:19:17 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 15:19:17 WARN c.n.s.JsonMetaSchema(newValidator):338 - {} - Unknown keyword order - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
source - 2024-05-21 15:19:17 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 15:19:17 WARN c.n.s.JsonMetaSchema(newValidator):338 - {} - Unknown keyword examples - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
source - 2024-05-21 15:19:17 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 15:19:17 WARN c.n.s.JsonMetaSchema(newValidator):338 - {} - Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
source - 2024-05-21 15:19:17 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 15:19:17 WARN c.n.s.JsonMetaSchema(newValidator):338 - {} - Unknown keyword multiline - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
source - 2024-05-21 15:19:18 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 15:19:18 INFO i.a.i.b.s.SshTunnel(getInstance):190 - {} - Starting connection with method: NO_TUNNEL
source - 2024-05-21 15:19:18 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 15:19:18 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$toWriteConfig$0):116 - {} - Write config: WriteConfig{streamName=accounts, namespace=null, outputSchemaName=public, tmpTableName=_airbyte_tmp_twv_accounts, outputTableName=_airbyte_raw_accounts, syncMode=append}
source - 2024-05-21 15:19:18 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 15:19:18 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$toWriteConfig$0):116 - {} - Write config: WriteConfig{streamName=ad_campaign_analytics, namespace=null, outputSchemaName=public, tmpTableName=_airbyte_tmp_bny_ad_campaign_analytics, outputTableName=_airbyte_raw_ad_campaign_analytics, syncMode=append}
source - 2024-05-21 15:19:18 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 15:19:18 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$toWriteConfig$0):116 - {} - Write config: WriteConfig{streamName=ad_creative_analytics, namespace=null, outputSchemaName=public, tmpTableName=_airbyte_tmp_ake_ad_creative_analytics, outputTableName=_airbyte_raw_ad_creative_analytics, syncMode=append}
source - 2024-05-21 15:19:18 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 15:19:18 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$toWriteConfig$0):116 - {} - Write config: WriteConfig{streamName=ad_direct_sponsored_contents, namespace=null, outputSchemaName=public, tmpTableName=_airbyte_tmp_nat_ad_direct_sponsored_contents, outputTableName=_airbyte_raw_ad_direct_sponsored_contents, syncMode=append}
source - 2024-05-21 15:19:18 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 15:19:18 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$toWriteConfig$0):116 - {} - Write config: WriteConfig{streamName=campaign_groups, namespace=null, outputSchemaName=public, tmpTableName=_airbyte_tmp_kvv_campaign_groups, outputTableName=_airbyte_raw_campaign_groups, syncMode=append}
source - 2024-05-21 15:19:18 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 15:19:18 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$toWriteConfig$0):116 - {} - Write config: WriteConfig{streamName=campaigns, namespace=null, outputSchemaName=public, tmpTableName=_airbyte_tmp_byc_campaigns, outputTableName=_airbyte_raw_campaigns, syncMode=append}
source - 2024-05-21 15:19:18 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 15:19:18 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$toWriteConfig$0):116 - {} - Write config: WriteConfig{streamName=creatives, namespace=null, outputSchemaName=public, tmpTableName=_airbyte_tmp_kxb_creatives, outputTableName=_airbyte_raw_creatives, syncMode=append}
source - 2024-05-21 15:19:18 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 15:19:18 INFO i.a.i.d.b.BufferedStreamConsumer(startTracked):143 - {} - class io.airbyte.integrations.destination.buffered_stream_consumer.BufferedStreamConsumer started.
source - 2024-05-21 15:19:18 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 15:19:18 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onStartFunction$1):139 - {} - Preparing tmp tables in destination started for 7 streams
source - 2024-05-21 15:19:18 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 15:19:18 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onStartFunction$1):143 - {} - Preparing tmp table in destination started for stream accounts. schema: public, tmp table name: _airbyte_tmp_twv_accounts
source - 2024-05-21 15:19:19 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 15:19:19 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onStartFunction$1):143 - {} - Preparing tmp table in destination started for stream ad_campaign_analytics. schema: public, tmp table name: _airbyte_tmp_bny_ad_campaign_analytics
source - 2024-05-21 15:19:19 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 15:19:19 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onStartFunction$1):143 - {} - Preparing tmp table in destination started for stream ad_creative_analytics. schema: public, tmp table name: _airbyte_tmp_ake_ad_creative_analytics
source - 2024-05-21 15:19:19 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 15:19:19 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onStartFunction$1):143 - {} - Preparing tmp table in destination started for stream ad_direct_sponsored_contents. schema: public, tmp table name: _airbyte_tmp_nat_ad_direct_sponsored_contents
source - 2024-05-21 15:19:19 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 15:19:19 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onStartFunction$1):143 - {} - Preparing tmp table in destination started for stream campaign_groups. schema: public, tmp table name: _airbyte_tmp_kvv_campaign_groups
source - 2024-05-21 15:19:19 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 15:19:19 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onStartFunction$1):143 - {} - Preparing tmp table in destination started for stream campaigns. schema: public, tmp table name: _airbyte_tmp_byc_campaigns
source - 2024-05-21 15:19:19 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 15:19:19 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onStartFunction$1):143 - {} - Preparing tmp table in destination started for stream creatives. schema: public, tmp table name: _airbyte_tmp_kxb_creatives
source - 2024-05-21 15:19:19 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 15:19:19 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onStartFunction$1):149 - {} - Preparing tables in destination completed.
2024-05-21 16:31:31 INFO () TemporalAttemptExecution(lambda$getCancellationChecker$2):181 - Running sync worker cancellation...
2024-05-21 16:31:31 INFO () DefaultReplicationWorker(cancel):250 - Cancelling replication worker...
source - 2024-05-21 16:31:37 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:31:37 INFO i.a.i.b.IntegrationRunner(run):96 - {} - Running integration: io.airbyte.integrations.base.ssh.SshWrappedDestination
source - 2024-05-21 16:31:37 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:31:37 INFO i.a.i.b.IntegrationCliParser(parseOptions):135 - {} - integration args: {catalog=destination_catalog.json, write=null, config=destination_config.json}
source - 2024-05-21 16:31:37 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:31:37 INFO i.a.i.b.IntegrationRunner(run):100 - {} - Command: WRITE
source - 2024-05-21 16:31:37 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:31:37 INFO i.a.i.b.IntegrationRunner(run):101 - {} - Integration config: IntegrationConfig{command=WRITE, configPath='destination_config.json', catalogPath='destination_catalog.json', statePath='null'}
source - 2024-05-21 16:31:38 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:31:38 WARN c.n.s.JsonMetaSchema(newValidator):338 - {} - Unknown keyword order - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
source - 2024-05-21 16:31:38 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:31:38 WARN c.n.s.JsonMetaSchema(newValidator):338 - {} - Unknown keyword examples - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
source - 2024-05-21 16:31:38 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:31:38 WARN c.n.s.JsonMetaSchema(newValidator):338 - {} - Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
source - 2024-05-21 16:31:38 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:31:38 WARN c.n.s.JsonMetaSchema(newValidator):338 - {} - Unknown keyword multiline - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
source - 2024-05-21 16:31:38 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:31:38 INFO i.a.i.b.s.SshTunnel(getInstance):190 - {} - Starting connection with method: NO_TUNNEL
source - 2024-05-21 16:31:39 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:31:39 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$toWriteConfig$0):116 - {} - Write config: WriteConfig{streamName=accounts, namespace=null, outputSchemaName=public, tmpTableName=_airbyte_tmp_lrs_accounts, outputTableName=_airbyte_raw_accounts, syncMode=append}
source - 2024-05-21 16:31:39 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:31:39 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$toWriteConfig$0):116 - {} - Write config: WriteConfig{streamName=ad_campaign_analytics, namespace=null, outputSchemaName=public, tmpTableName=_airbyte_tmp_kuu_ad_campaign_analytics, outputTableName=_airbyte_raw_ad_campaign_analytics, syncMode=append}
source - 2024-05-21 16:31:39 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:31:39 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$toWriteConfig$0):116 - {} - Write config: WriteConfig{streamName=ad_creative_analytics, namespace=null, outputSchemaName=public, tmpTableName=_airbyte_tmp_asw_ad_creative_analytics, outputTableName=_airbyte_raw_ad_creative_analytics, syncMode=append}
source - 2024-05-21 16:31:39 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:31:39 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$toWriteConfig$0):116 - {} - Write config: WriteConfig{streamName=ad_direct_sponsored_contents, namespace=null, outputSchemaName=public, tmpTableName=_airbyte_tmp_ylj_ad_direct_sponsored_contents, outputTableName=_airbyte_raw_ad_direct_sponsored_contents, syncMode=append}
source - 2024-05-21 16:31:39 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:31:39 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$toWriteConfig$0):116 - {} - Write config: WriteConfig{streamName=campaign_groups, namespace=null, outputSchemaName=public, tmpTableName=_airbyte_tmp_gge_campaign_groups, outputTableName=_airbyte_raw_campaign_groups, syncMode=append}
source - 2024-05-21 16:31:39 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:31:39 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$toWriteConfig$0):116 - {} - Write config: WriteConfig{streamName=campaigns, namespace=null, outputSchemaName=public, tmpTableName=_airbyte_tmp_sjh_campaigns, outputTableName=_airbyte_raw_campaigns, syncMode=append}
source - 2024-05-21 16:31:39 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:31:39 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$toWriteConfig$0):116 - {} - Write config: WriteConfig{streamName=creatives, namespace=null, outputSchemaName=public, tmpTableName=_airbyte_tmp_cem_creatives, outputTableName=_airbyte_raw_creatives, syncMode=append}
source - 2024-05-21 16:31:39 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:31:39 INFO i.a.i.d.b.BufferedStreamConsumer(startTracked):143 - {} - class io.airbyte.integrations.destination.buffered_stream_consumer.BufferedStreamConsumer started.
source - 2024-05-21 16:31:39 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:31:39 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onStartFunction$1):139 - {} - Preparing tmp tables in destination started for 7 streams
source - 2024-05-21 16:31:39 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:31:39 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onStartFunction$1):143 - {} - Preparing tmp table in destination started for stream accounts. schema: public, tmp table name: _airbyte_tmp_lrs_accounts
source - 2024-05-21 16:31:40 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:31:40 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onStartFunction$1):143 - {} - Preparing tmp table in destination started for stream ad_campaign_analytics. schema: public, tmp table name: _airbyte_tmp_kuu_ad_campaign_analytics
source - 2024-05-21 16:31:40 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:31:40 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onStartFunction$1):143 - {} - Preparing tmp table in destination started for stream ad_creative_analytics. schema: public, tmp table name: _airbyte_tmp_asw_ad_creative_analytics
source - 2024-05-21 16:31:40 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:31:40 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onStartFunction$1):143 - {} - Preparing tmp table in destination started for stream ad_direct_sponsored_contents. schema: public, tmp table name: _airbyte_tmp_ylj_ad_direct_sponsored_contents
source - 2024-05-21 16:31:40 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:31:40 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onStartFunction$1):143 - {} - Preparing tmp table in destination started for stream campaign_groups. schema: public, tmp table name: _airbyte_tmp_gge_campaign_groups
source - 2024-05-21 16:31:40 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:31:40 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onStartFunction$1):143 - {} - Preparing tmp table in destination started for stream campaigns. schema: public, tmp table name: _airbyte_tmp_sjh_campaigns
source - 2024-05-21 16:31:40 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:31:40 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onStartFunction$1):143 - {} - Preparing tmp table in destination started for stream creatives. schema: public, tmp table name: _airbyte_tmp_cem_creatives
source - 2024-05-21 16:31:40 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:31:40 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onStartFunction$1):149 - {} - Preparing tables in destination completed.
2024-05-21 16:31:41 INFO () DefaultReplicationWorker(cancel):258 - Cancelling destination...
2024-05-21 16:31:41 INFO () DefaultAirbyteDestination(cancel):121 - Attempting to cancel destination process...
2024-05-21 16:31:41 INFO () DefaultAirbyteDestination(cancel):126 - Destination process exists, cancelling...
2024-05-21 16:31:41 WARN () LineGobbler(voidCall):86 - airbyte-destination gobbler IOException: Stream closed. Typically happens when cancelling a job.
2024-05-21 16:31:41 INFO () DefaultAirbyteDestination(cancel):128 - Cancelled destination process!
2024-05-21 16:31:41 INFO () DefaultReplicationWorker(cancel):265 - Cancelling source...
2024-05-21 16:31:41 INFO () DefaultAirbyteSource(cancel):129 - Attempting to cancel source process...
2024-05-21 16:31:41 INFO () DefaultAirbyteSource(cancel):134 - Source process exists, cancelling...
2024-05-21 16:31:41 INFO () DefaultReplicationWorker(run):121 - Source thread complete.
2024-05-21 16:31:41 INFO () DefaultReplicationWorker(run):122 - Waiting for destination thread to join.
2024-05-21 16:31:41 INFO () DefaultReplicationWorker(run):124 - Destination thread complete.
2024-05-21 16:31:41 INFO () DefaultAirbyteSource(cancel):136 - Cancelled source process!
2024-05-21 16:31:41 INFO () TemporalAttemptExecution(lambda$getCancellationChecker$2):185 - Interrupting worker thread...
2024-05-21 16:31:41 INFO () TemporalAttemptExecution(lambda$getCancellationChecker$2):188 - Cancelling completable future...
2024-05-21 16:31:41 WARN () CancellationHandler$TemporalCancellationHandler(checkAndHandleCancellation):51 - Job either timeout-ed or was cancelled.
destination - 2024-05-21 16:31:41 INFO () TemporalAttemptExecution(get):115 - Stopping cancellation check scheduling...
2024-05-21 16:31:41 WARN () CancellationHandler$TemporalCancellationHandler(checkAndHandleCancellation):51 - Job either timeout-ed or was cancelled.
2024-05-21 16:31:41 ERROR () DefaultReplicationWorker(run):128 - Sync worker failed.
io.airbyte.workers.WorkerException: Source process exit with code 143. This warning is normal if the job was cancelled.
    at io.airbyte.workers.protocols.airbyte.DefaultAirbyteSource.close(DefaultAirbyteSource.java:123) ~[io.airbyte-airbyte-workers-0.30.25-alpha.jar:?]
    at io.airbyte.workers.DefaultReplicationWorker.run(DefaultReplicationWorker.java:126) ~[io.airbyte-airbyte-workers-0.30.25-alpha.jar:?]
    at io.airbyte.workers.DefaultReplicationWorker.run(DefaultReplicationWorker.java:32) ~[io.airbyte-airbyte-workers-0.30.25-alpha.jar:?]
    at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$1(TemporalAttemptExecution.java:145) ~[io.airbyte-airbyte-workers-0.30.25-alpha.jar:?]
    at java.lang.Thread.run(Thread.java:832) [?:?]
    Suppressed: io.airbyte.workers.WorkerException: Destination process exit with code 1. This warning is normal if the job was cancelled.
        at io.airbyte.workers.protocols.airbyte.DefaultAirbyteDestination.close(DefaultAirbyteDestination.java:115) ~[io.airbyte-airbyte-workers-0.30.25-alpha.jar:?]
        at io.airbyte.workers.DefaultReplicationWorker.run(DefaultReplicationWorker.java:101) ~[io.airbyte-airbyte-workers-0.30.25-alpha.jar:?]
        at io.airbyte.workers.DefaultReplicationWorker.run(DefaultReplicationWorker.java:32) ~[io.airbyte-airbyte-workers-0.30.25-alpha.jar:?]
        at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$1(TemporalAttemptExecution.java:145) ~[io.airbyte-airbyte-workers-0.30.25-alpha.jar:?]
        at java.lang.Thread.run(Thread.java:832) [?:?]
destination - 2024-05-21 16:31:41 WARN () POJOActivityTaskHandler$POJOActivityImplementation(execute):243 - Activity failure. ActivityId=f9445118-f6e0-3e5c-bb4a-cd2033adb143, activityType=Replicate, attempt=1
java.util.concurrent.CancellationException: null
    at java.util.concurrent.CompletableFuture.cancel(CompletableFuture.java:2468) ~[?:?]
    at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getCancellationChecker$2(TemporalAttemptExecution.java:191) ~[io.airbyte-airbyte-workers-0.30.25-alpha.jar:?]
    at io.airbyte.workers.temporal.CancellationHandler$TemporalCancellationHandler.checkAndHandleCancellation(CancellationHandler.java:50) ~[io.airbyte-airbyte-workers-0.30.25-alpha.jar:?]
    at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getCancellationChecker$3(TemporalAttemptExecution.java:194) ~[io.airbyte-airbyte-workers-0.30.25-alpha.jar:?]
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?]
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) ~[?:?]
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) ~[?:?]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?]
    at java.lang.Thread.run(Thread.java:832) [?:?]
2024-05-21 16:31:41 INFO () DefaultReplicationWorker(run):152 - sync summary: io.airbyte.config.ReplicationAttemptSummary@1babad4f[status=cancelled,recordsSynced=0,bytesSynced=0,startTime=1716304751653,endTime=1716309101934]
2024-05-21 16:31:41 INFO () DefaultReplicationWorker(run):161 - Source did not output any state messages
2024-05-21 16:31:41 WARN () DefaultReplicationWorker(run):172 - State capture: No state retained.
source - 2024-05-21 16:34:42 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:34:42 INFO i.a.i.b.IntegrationRunner(run):96 - {} - Running integration: io.airbyte.integrations.base.ssh.SshWrappedDestination
source - 2024-05-21 16:34:42 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:34:42 INFO i.a.i.b.IntegrationCliParser(parseOptions):135 - {} - integration args: {catalog=destination_catalog.json, write=null, config=destination_config.json}
source - 2024-05-21 16:34:42 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:34:42 INFO i.a.i.b.IntegrationRunner(run):100 - {} - Command: WRITE
source - 2024-05-21 16:34:42 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:34:42 INFO i.a.i.b.IntegrationRunner(run):101 - {} - Integration config: IntegrationConfig{command=WRITE, configPath='destination_config.json', catalogPath='destination_catalog.json', statePath='null'}
source - 2024-05-21 16:34:42 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:34:42 WARN c.n.s.JsonMetaSchema(newValidator):338 - {} - Unknown keyword order - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
source - 2024-05-21 16:34:42 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:34:42 WARN c.n.s.JsonMetaSchema(newValidator):338 - {} - Unknown keyword examples - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
source - 2024-05-21 16:34:42 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:34:42 WARN c.n.s.JsonMetaSchema(newValidator):338 - {} - Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
source - 2024-05-21 16:34:42 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:34:42 WARN c.n.s.JsonMetaSchema(newValidator):338 - {} - Unknown keyword multiline - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
source - 2024-05-21 16:34:43 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:34:43 INFO i.a.i.b.s.SshTunnel(getInstance):190 - {} - Starting connection with method: NO_TUNNEL
source - 2024-05-21 16:34:43 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:34:43 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$toWriteConfig$0):116 - {} - Write config: WriteConfig{streamName=accounts, namespace=null, outputSchemaName=public, tmpTableName=_airbyte_tmp_uti_accounts, outputTableName=_airbyte_raw_accounts, syncMode=append}
source - 2024-05-21 16:34:43 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:34:43 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$toWriteConfig$0):116 - {} - Write config: WriteConfig{streamName=ad_campaign_analytics, namespace=null, outputSchemaName=public, tmpTableName=_airbyte_tmp_fdf_ad_campaign_analytics, outputTableName=_airbyte_raw_ad_campaign_analytics, syncMode=append}
source - 2024-05-21 16:34:43 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:34:43 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$toWriteConfig$0):116 - {} - Write config: WriteConfig{streamName=ad_creative_analytics, namespace=null, outputSchemaName=public, tmpTableName=_airbyte_tmp_rji_ad_creative_analytics, outputTableName=_airbyte_raw_ad_creative_analytics, syncMode=append}
source - 2024-05-21 16:34:43 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:34:43 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$toWriteConfig$0):116 - {} - Write config: WriteConfig{streamName=ad_direct_sponsored_contents, namespace=null, outputSchemaName=public, tmpTableName=_airbyte_tmp_jtf_ad_direct_sponsored_contents, outputTableName=_airbyte_raw_ad_direct_sponsored_contents, syncMode=append}
source - 2024-05-21 16:34:43 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:34:43 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$toWriteConfig$0):116 - {} - Write config: WriteConfig{streamName=campaign_groups, namespace=null, outputSchemaName=public, tmpTableName=_airbyte_tmp_fqp_campaign_groups, outputTableName=_airbyte_raw_campaign_groups, syncMode=append}
source - 2024-05-21 16:34:43 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:34:43 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$toWriteConfig$0):116 - {} - Write config: WriteConfig{streamName=campaigns, namespace=null, outputSchemaName=public, tmpTableName=_airbyte_tmp_snt_campaigns, outputTableName=_airbyte_raw_campaigns, syncMode=append}
source - 2024-05-21 16:34:43 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:34:43 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$toWriteConfig$0):116 - {} - Write config: WriteConfig{streamName=creatives, namespace=null, outputSchemaName=public, tmpTableName=_airbyte_tmp_szd_creatives, outputTableName=_airbyte_raw_creatives, syncMode=append}
source - 2024-05-21 16:34:43 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:34:43 INFO i.a.i.d.b.BufferedStreamConsumer(startTracked):143 - {} - class io.airbyte.integrations.destination.buffered_stream_consumer.BufferedStreamConsumer started.
source - 2024-05-21 16:34:43 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:34:43 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onStartFunction$1):139 - {} - Preparing tmp tables in destination started for 7 streams
source - 2024-05-21 16:34:43 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:34:43 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onStartFunction$1):143 - {} - Preparing tmp table in destination started for stream accounts. schema: public, tmp table name: _airbyte_tmp_uti_accounts
source - 2024-05-21 16:34:44 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:34:44 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onStartFunction$1):143 - {} - Preparing tmp table in destination started for stream ad_campaign_analytics. schema: public, tmp table name: _airbyte_tmp_fdf_ad_campaign_analytics
source - 2024-05-21 16:34:44 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:34:44 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onStartFunction$1):143 - {} - Preparing tmp table in destination started for stream ad_creative_analytics. schema: public, tmp table name: _airbyte_tmp_rji_ad_creative_analytics
source - 2024-05-21 16:34:44 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:34:44 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onStartFunction$1):143 - {} - Preparing tmp table in destination started for stream ad_direct_sponsored_contents. schema: public, tmp table name: _airbyte_tmp_jtf_ad_direct_sponsored_contents
source - 2024-05-21 16:34:44 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:34:44 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onStartFunction$1):143 - {} - Preparing tmp table in destination started for stream campaign_groups. schema: public, tmp table name: _airbyte_tmp_fqp_campaign_groups
source - 2024-05-21 16:34:44 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:34:44 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onStartFunction$1):143 - {} - Preparing tmp table in destination started for stream campaigns. schema: public, tmp table name: _airbyte_tmp_snt_campaigns
source - 2024-05-21 16:34:44 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:34:44 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onStartFunction$1):143 - {} - Preparing tmp table in destination started for stream creatives. schema: public, tmp table name: _airbyte_tmp_szd_creatives
source - 2024-05-21 16:34:44 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:34:44 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onStartFunction$1):149 - {} - Preparing tables in destination completed.

=== Second Failed Sync ===
2024-05-21 16:34:38 INFO () WorkerRun(call):49 - Executing worker wrapper. Airbyte version: 0.30.25-alpha
2024-05-21 16:34:38 INFO () TemporalAttemptExecution(get):94 - Executing worker wrapper. Airbyte version: 0.30.25-alpha
2024-05-21 16:34:38 WARN () Databases(createPostgresDatabaseWithRetry):38 - Waiting for database to become available...
2024-05-21 16:34:38 INFO () JobsDatabaseInstance(lambda$static$2):25 - Testing if jobs database is ready...
2024-05-21 16:34:38 INFO () Databases(createPostgresDatabaseWithRetry):55 - Database available!
2024-05-21 16:34:38 INFO () DefaultReplicationWorker(run):82 - start sync worker. job id: 3 attempt id: 0
2024-05-21 16:34:38 INFO () DefaultReplicationWorker(run):91 - configured sync modes: {null.accounts=full_refresh - append, null.ad_creative_analytics=full_refresh - append, null.campaign_groups=full_refresh - append, null.creatives=full_refresh - append, null.ad_direct_sponsored_contents=full_refresh - append, null.campaigns=full_refresh - append, null.ad_campaign_analytics=full_refresh - append}
2024-05-21 16:34:38 INFO () DefaultAirbyteDestination(start):65 - Running destination...
2024-05-21 16:34:38 INFO () LineGobbler(voidCall):82 - Checking if airbyte/destination-postgres:0.3.11 exists...
2024-05-21 16:34:38 INFO () DockerProcessFactory(create):127 - Preparing command: docker run --rm --init -i -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -w /data/3/0 --network host --log-driver none airbyte/destination-postgres:0.3.11 write --config destination_config.json --catalog destination_catalog.json
2024-05-21 16:34:38 INFO () LineGobbler(voidCall):82 - Checking if airbyte/source-linkedin-ads:0.1.1 exists...
2024-05-21 16:34:38 INFO () DockerProcessFactory(create):127 - Preparing command: docker run --rm --init -i -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -w /data/3/0 --network host --log-driver none airbyte/source-linkedin-ads:0.1.1 read --config source_config.json --catalog source_catalog.json
2024-05-21 16:34:39 INFO () DefaultReplicationWorker(lambda$getDestinationOutputRunnable$3):226 - Destination output thread started.
2024-05-21 16:34:39 INFO () DefaultReplicationWorker(run):119 - Waiting for source thread to join.
2024-05-21 16:34:39 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):190 - Replication thread started.
2024-05-21 16:34:42 INFO () DefaultAirbyteStreamFactory(lambda$create$0):62 - 2024-05-21 16:34:41 INFO i.a.i.d.p.PostgresDestination(main):85 - {} - starting destination: class io.airbyte.integrations.destination.postgres.PostgresDestination

Contribute

marcosmarxm commented 4 months ago

If you're reporting problem related to postgres destination please fill the form provinding information about that connector.