search

airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
15.53k stars 4k forks source link

Sync Error: Possibly too small block size used. Please try to increase it #28502

Closed KrishnaPG closed 12 months ago

KrishnaPG commented 1 year ago

What method are you using to run Airbyte?

Docker

Platform Version or Helm Chart Version

0.50.5

What step the error happened?

During the Sync

Revelant information

image

Relevant log output

2023-07-20 01:18:24 INFO i.a.w.t.TemporalAttemptExecution(get):136 - Docker volume job log path: /tmp/workspace/16/0/logs.log
2023-07-20 01:18:24 INFO i.a.w.t.TemporalAttemptExecution(get):141 - Executing worker wrapper. Airbyte version: 0.50.5
2023-07-20 01:18:24 INFO i.a.a.c.AirbyteApiClient(retryWithJitterThrows):229 - Attempt 0 to save workflow id for cancellation
2023-07-20 01:18:24 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable SIDECAR_KUBE_CPU_LIMIT: '2.0'
2023-07-20 01:18:24 INFO i.a.c.i.LineGobbler(voidCall):149 - 
2023-07-20 01:18:24 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable SOCAT_KUBE_CPU_LIMIT: '2.0'
2023-07-20 01:18:24 INFO i.a.c.i.LineGobbler(voidCall):149 - ----- START CHECK -----
2023-07-20 01:18:24 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable SIDECAR_KUBE_CPU_REQUEST: '0.1'
2023-07-20 01:18:24 INFO i.a.c.i.LineGobbler(voidCall):149 - 
2023-07-20 01:18:24 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable SOCAT_KUBE_CPU_REQUEST: '0.1'
2023-07-20 01:18:24 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable LAUNCHDARKLY_KEY: ''
2023-07-20 01:18:24 INFO i.a.c.i.LineGobbler(voidCall):149 - Checking if airbyte/source-s3:3.1.0 exists...
2023-07-20 01:18:24 INFO i.a.c.i.LineGobbler(voidCall):149 - airbyte/source-s3:3.1.0 was found locally.
2023-07-20 01:18:24 INFO i.a.w.p.DockerProcessFactory(create):139 - Creating docker container = source-s3-check-16-0-hdezg with resources io.airbyte.config.ResourceRequirements@67020bfb[cpuRequest=,cpuLimit=,memoryRequest=,memoryLimit=,additionalProperties={}] and allowedHosts io.airbyte.config.AllowedHosts@1d720b43[hosts=[*.s3.amazonaws.com, *.datadoghq.com, *.datadoghq.eu, *.sentry.io],additionalProperties={}]
2023-07-20 01:18:24 INFO i.a.w.p.DockerProcessFactory(create):192 - Preparing command: docker run --rm --init -i -w /data/16/0 --log-driver none --name source-s3-check-16-0-hdezg --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e WORKER_CONNECTOR_IMAGE=airbyte/source-s3:3.1.0 -e AUTO_DETECT_SCHEMA=true -e LAUNCHDARKLY_KEY= -e SOCAT_KUBE_CPU_REQUEST=0.1 -e SOCAT_KUBE_CPU_LIMIT=2.0 -e USE_STREAM_CAPABLE_STATE=true -e FIELD_SELECTION_WORKSPACES= -e WORKER_ENVIRONMENT=DOCKER -e AIRBYTE_ROLE= -e APPLY_FIELD_SELECTION=false -e WORKER_JOB_ATTEMPT=0 -e OTEL_COLLECTOR_ENDPOINT=http://host.docker.internal:4317 -e FEATURE_FLAG_CLIENT=config -e AIRBYTE_VERSION=0.50.5 -e WORKER_JOB_ID=16 airbyte/source-s3:3.1.0 check --config source_config.json
2023-07-20 01:18:24 INFO i.a.w.i.VersionedAirbyteStreamFactory(create):177 - Reading messages from protocol version 0.2.0
2023-07-20 01:18:25 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):312 - initialised stream with format: 
2023-07-20 01:18:25 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):312 - initialised stream with format: {'encoding': 'utf8', 'filetype': 'csv', 'delimiter': ',', 'block_size': 10000, 'quote_char': '"', 'double_quote': True, 'infer_datatypes': True, 'newlines_in_values': False}
2023-07-20 01:18:25 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):312 - Iterating S3 bucket 'hmda'
2023-07-20 01:18:25 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):312 - Iterating S3 bucket 'hmda'
2023-07-20 01:18:26 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):312 - Reading from file: Key: food_inspections/Food_Inspections.csv, LastModified: 2023-07-19T04:16:43.342000+00:00, Size: 273.6943Mb
2023-07-20 01:18:26 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):312 - Check succeeded
2023-07-20 01:18:27 INFO i.a.w.g.DefaultCheckConnectionWorker(run):115 - Check connection job received output: io.airbyte.config.StandardCheckConnectionOutput@6ebfa865[status=succeeded,message=<null>,additionalProperties={}]
2023-07-20 01:18:27 INFO i.a.w.t.TemporalAttemptExecution(get):163 - Stopping cancellation check scheduling...
2023-07-20 01:18:27 INFO i.a.c.i.LineGobbler(voidCall):149 - 
2023-07-20 01:18:27 INFO i.a.c.i.LineGobbler(voidCall):149 - ----- END CHECK -----
2023-07-20 01:18:27 INFO i.a.c.i.LineGobbler(voidCall):149 - 
2023-07-20 01:18:27 INFO i.a.w.t.TemporalAttemptExecution(get):136 - Docker volume job log path: /tmp/workspace/16/0/logs.log
2023-07-20 01:18:27 INFO i.a.w.t.TemporalAttemptExecution(get):141 - Executing worker wrapper. Airbyte version: 0.50.5
2023-07-20 01:18:27 INFO i.a.a.c.AirbyteApiClient(retryWithJitterThrows):229 - Attempt 0 to save workflow id for cancellation
2023-07-20 01:18:27 INFO i.a.c.i.LineGobbler(voidCall):149 - 
2023-07-20 01:18:27 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable SIDECAR_KUBE_CPU_LIMIT: '2.0'
2023-07-20 01:18:27 INFO i.a.c.i.LineGobbler(voidCall):149 - ----- START CHECK -----
2023-07-20 01:18:27 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable SOCAT_KUBE_CPU_LIMIT: '2.0'
2023-07-20 01:18:27 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable SIDECAR_KUBE_CPU_REQUEST: '0.1'
2023-07-20 01:18:27 INFO i.a.c.i.LineGobbler(voidCall):149 - 
2023-07-20 01:18:27 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable SOCAT_KUBE_CPU_REQUEST: '0.1'
2023-07-20 01:18:27 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable LAUNCHDARKLY_KEY: ''
2023-07-20 01:18:27 INFO i.a.c.i.LineGobbler(voidCall):149 - Checking if airbyte/destination-clickhouse:0.2.5 exists...
2023-07-20 01:18:27 INFO i.a.c.i.LineGobbler(voidCall):149 - airbyte/destination-clickhouse:0.2.5 was found locally.
2023-07-20 01:18:27 INFO i.a.w.p.DockerProcessFactory(create):139 - Creating docker container = destination-clickhouse-check-16-0-gzmcz with resources io.airbyte.config.ResourceRequirements@105f710b[cpuRequest=,cpuLimit=,memoryRequest=,memoryLimit=,additionalProperties={}] and allowedHosts null
2023-07-20 01:18:27 INFO i.a.w.p.DockerProcessFactory(create):192 - Preparing command: docker run --rm --init -i -w /data/16/0 --log-driver none --name destination-clickhouse-check-16-0-gzmcz --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e WORKER_CONNECTOR_IMAGE=airbyte/destination-clickhouse:0.2.5 -e AUTO_DETECT_SCHEMA=true -e LAUNCHDARKLY_KEY= -e SOCAT_KUBE_CPU_REQUEST=0.1 -e SOCAT_KUBE_CPU_LIMIT=2.0 -e USE_STREAM_CAPABLE_STATE=true -e FIELD_SELECTION_WORKSPACES= -e WORKER_ENVIRONMENT=DOCKER -e AIRBYTE_ROLE= -e APPLY_FIELD_SELECTION=false -e WORKER_JOB_ATTEMPT=0 -e OTEL_COLLECTOR_ENDPOINT=http://host.docker.internal:4317 -e FEATURE_FLAG_CLIENT=config -e AIRBYTE_VERSION=0.50.5 -e WORKER_JOB_ID=16 airbyte/destination-clickhouse:0.2.5 check --config source_config.json
2023-07-20 01:18:27 INFO i.a.w.i.VersionedAirbyteStreamFactory(create):177 - Reading messages from protocol version 0.2.0
2023-07-20 01:18:29 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):312 - INFO i.a.i.d.c.ClickhouseDestination(main):113 starting destination: class io.airbyte.integrations.destination.clickhouse.ClickhouseDestination
2023-07-20 01:18:29 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):312 - INFO i.a.i.b.IntegrationCliParser(parseOptions):126 integration args: {check=null, config=source_config.json}
2023-07-20 01:18:29 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):312 - INFO i.a.i.b.IntegrationRunner(runInternal):106 Running integration: io.airbyte.integrations.base.ssh.SshWrappedDestination
2023-07-20 01:18:29 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):312 - INFO i.a.i.b.IntegrationRunner(runInternal):107 Command: CHECK
2023-07-20 01:18:29 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):312 - INFO i.a.i.b.IntegrationRunner(runInternal):108 Integration config: IntegrationConfig{command=CHECK, configPath='source_config.json', catalogPath='null', statePath='null'}
2023-07-20 01:18:29 WARN i.a.w.i.VersionedAirbyteStreamFactory(internalLog):309 - WARN c.n.s.JsonMetaSchema(newValidator):278 Unknown keyword order - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2023-07-20 01:18:29 WARN i.a.w.i.VersionedAirbyteStreamFactory(internalLog):309 - WARN c.n.s.JsonMetaSchema(newValidator):278 Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2023-07-20 01:18:29 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):312 - INFO i.a.i.b.s.SshTunnel(getInstance):204 Starting connection with method: NO_TUNNEL
2023-07-20 01:18:29 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):312 - INFO c.z.h.HikariDataSource(<init>):80 HikariPool-1 - Starting...
2023-07-20 01:18:29 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):312 - INFO c.z.h.HikariDataSource(<init>):82 HikariPool-1 - Start completed.
2023-07-20 01:18:30 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):312 - INFO c.z.h.HikariDataSource(close):350 HikariPool-1 - Shutdown initiated...
2023-07-20 01:18:30 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):312 - INFO c.z.h.HikariDataSource(close):352 HikariPool-1 - Shutdown completed.
2023-07-20 01:18:30 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):312 - INFO i.a.i.b.IntegrationRunner(runInternal):195 Completed integration: io.airbyte.integrations.base.ssh.SshWrappedDestination
2023-07-20 01:18:30 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):312 - INFO i.a.i.d.c.ClickhouseDestination(main):115 completed destination: class io.airbyte.integrations.destination.clickhouse.ClickhouseDestination
2023-07-20 01:18:30 INFO i.a.w.g.DefaultCheckConnectionWorker(run):115 - Check connection job received output: io.airbyte.config.StandardCheckConnectionOutput@684c340c[status=succeeded,message=<null>,additionalProperties={}]
2023-07-20 01:18:30 INFO i.a.w.t.TemporalAttemptExecution(get):163 - Stopping cancellation check scheduling...
2023-07-20 01:18:30 INFO i.a.c.i.LineGobbler(voidCall):149 - 
2023-07-20 01:18:30 INFO i.a.c.i.LineGobbler(voidCall):149 - ----- END CHECK -----
2023-07-20 01:18:30 INFO i.a.c.i.LineGobbler(voidCall):149 - 
2023-07-20 01:18:30 INFO i.a.a.c.AirbyteApiClient(retryWithJitterThrows):229 - Attempt 0 to get state
2023-07-20 01:18:30 INFO i.a.w.h.NormalizationInDestinationHelper(shouldNormalizeInDestination):52 - Requires Normalization: true, Normalization Supported: false, Feature Flag Enabled: false
2023-07-20 01:18:30 INFO i.a.a.c.AirbyteApiClient(retryWithJitterThrows):229 - Attempt 0 to set attempt sync config
2023-07-20 01:18:30 INFO i.a.c.t.s.DefaultTaskQueueMapper(getTaskQueue):31 - Called DefaultTaskQueueMapper getTaskQueue for geography auto
2023-07-20 01:18:30 INFO i.a.w.t.TemporalAttemptExecution(get):136 - Docker volume job log path: /tmp/workspace/16/0/logs.log
2023-07-20 01:18:30 INFO i.a.w.t.TemporalAttemptExecution(get):141 - Executing worker wrapper. Airbyte version: 0.50.5
2023-07-20 01:18:30 INFO i.a.a.c.AirbyteApiClient(retryWithJitterThrows):229 - Attempt 0 to save workflow id for cancellation
2023-07-20 01:18:30 INFO i.a.a.c.AirbyteApiClient(retryWithJitterThrows):229 - Attempt 0 to get the source definition for feature flag checks
2023-07-20 01:18:30 INFO i.a.a.c.AirbyteApiClient(retryWithJitterThrows):229 - Attempt 0 to get the source definition
2023-07-20 01:18:30 INFO i.a.w.g.ReplicationWorkerFactory(maybeEnableConcurrentStreamReads):166 - Concurrent stream read enabled? false
2023-07-20 01:18:30 INFO i.a.w.g.ReplicationWorkerFactory(create):127 - Setting up source...
2023-07-20 01:18:30 INFO i.a.w.g.ReplicationWorkerFactory(create):134 - Setting up destination...
2023-07-20 01:18:30 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable METRIC_CLIENT: ''
2023-07-20 01:18:30 WARN i.a.m.l.MetricClientFactory(initialize):60 - Metric client is already initialized to 
2023-07-20 01:18:30 INFO i.a.w.g.ReplicationWorkerFactory(create):146 - Setting up replication worker...
2023-07-20 01:18:30 INFO i.a.w.g.DefaultReplicationWorker(run):124 - start sync worker. job id: 16 attempt id: 0
2023-07-20 01:18:30 INFO i.a.w.g.DefaultReplicationWorker(run):129 - configured sync modes: {null.chicago_food_inspections=full_refresh - overwrite}
2023-07-20 01:18:30 INFO i.a.c.i.LineGobbler(voidCall):149 - 
2023-07-20 01:18:30 INFO i.a.c.i.LineGobbler(voidCall):149 - ----- START REPLICATION -----
2023-07-20 01:18:30 INFO i.a.w.g.DefaultReplicationWorker(run):141 - Committing states from replication activity
2023-07-20 01:18:30 INFO i.a.c.i.LineGobbler(voidCall):149 - 
2023-07-20 01:18:30 INFO i.a.w.g.DefaultReplicationWorker(run):143 - Committing stats from replication activity
2023-07-20 01:18:30 INFO i.a.w.i.DefaultAirbyteDestination(start):88 - Running destination...
2023-07-20 01:18:30 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable SIDECAR_KUBE_CPU_LIMIT: '2.0'
2023-07-20 01:18:30 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable SOCAT_KUBE_CPU_LIMIT: '2.0'
2023-07-20 01:18:30 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable SIDECAR_KUBE_CPU_REQUEST: '0.1'
2023-07-20 01:18:30 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable SOCAT_KUBE_CPU_REQUEST: '0.1'
2023-07-20 01:18:30 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable LAUNCHDARKLY_KEY: ''
2023-07-20 01:18:30 INFO i.a.c.i.LineGobbler(voidCall):149 - Checking if airbyte/destination-clickhouse:0.2.5 exists...
2023-07-20 01:18:30 INFO i.a.c.i.LineGobbler(voidCall):149 - airbyte/destination-clickhouse:0.2.5 was found locally.
2023-07-20 01:18:30 INFO i.a.w.p.DockerProcessFactory(create):139 - Creating docker container = destination-clickhouse-write-16-0-xtvok with resources io.airbyte.config.ResourceRequirements@53529f9c[cpuRequest=,cpuLimit=,memoryRequest=,memoryLimit=,additionalProperties={}] and allowedHosts null
2023-07-20 01:18:30 INFO i.a.w.p.DockerProcessFactory(create):192 - Preparing command: docker run --rm --init -i -w /data/16/0 --log-driver none --name destination-clickhouse-write-16-0-xtvok --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e WORKER_CONNECTOR_IMAGE=airbyte/destination-clickhouse:0.2.5 -e AUTO_DETECT_SCHEMA=true -e LAUNCHDARKLY_KEY= -e SOCAT_KUBE_CPU_REQUEST=0.1 -e SOCAT_KUBE_CPU_LIMIT=2.0 -e USE_STREAM_CAPABLE_STATE=true -e FIELD_SELECTION_WORKSPACES= -e WORKER_ENVIRONMENT=DOCKER -e AIRBYTE_ROLE= -e APPLY_FIELD_SELECTION=false -e WORKER_JOB_ATTEMPT=0 -e OTEL_COLLECTOR_ENDPOINT=http://host.docker.internal:4317 -e FEATURE_FLAG_CLIENT=config -e AIRBYTE_VERSION=0.50.5 -e WORKER_JOB_ID=16 airbyte/destination-clickhouse:0.2.5 write --config destination_config.json --catalog destination_catalog.json
2023-07-20 01:18:30 INFO i.a.w.i.VersionedAirbyteMessageBufferedWriterFactory(createWriter):41 - Writing messages to protocol version 0.2.0
2023-07-20 01:18:30 INFO i.a.w.i.VersionedAirbyteStreamFactory(create):177 - Reading messages from protocol version 0.2.0
2023-07-20 01:18:30 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable SIDECAR_KUBE_CPU_LIMIT: '2.0'
2023-07-20 01:18:30 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable SOCAT_KUBE_CPU_LIMIT: '2.0'
2023-07-20 01:18:30 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable SIDECAR_KUBE_CPU_REQUEST: '0.1'
2023-07-20 01:18:30 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable SOCAT_KUBE_CPU_REQUEST: '0.1'
2023-07-20 01:18:30 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable LAUNCHDARKLY_KEY: ''
2023-07-20 01:18:30 INFO i.a.c.i.LineGobbler(voidCall):149 - Checking if airbyte/source-s3:3.1.0 exists...
2023-07-20 01:18:30 INFO i.a.c.i.LineGobbler(voidCall):149 - airbyte/source-s3:3.1.0 was found locally.
2023-07-20 01:18:30 INFO i.a.w.p.DockerProcessFactory(create):139 - Creating docker container = source-s3-read-16-0-cleyr with resources io.airbyte.config.ResourceRequirements@3e4ef73d[cpuRequest=0.5,cpuLimit=,memoryRequest=,memoryLimit=,additionalProperties={}] and allowedHosts io.airbyte.config.AllowedHosts@6483a2fb[hosts=[*.s3.amazonaws.com, *.datadoghq.com, *.datadoghq.eu, *.sentry.io],additionalProperties={}]
2023-07-20 01:18:30 INFO i.a.w.p.DockerProcessFactory(create):192 - Preparing command: docker run --rm --init -i -w /data/16/0 --log-driver none --name source-s3-read-16-0-cleyr -e CONCURRENT_SOURCE_STREAM_READ=false --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e WORKER_CONNECTOR_IMAGE=airbyte/source-s3:3.1.0 -e AUTO_DETECT_SCHEMA=true -e LAUNCHDARKLY_KEY= -e SOCAT_KUBE_CPU_REQUEST=0.1 -e SOCAT_KUBE_CPU_LIMIT=2.0 -e USE_STREAM_CAPABLE_STATE=true -e FIELD_SELECTION_WORKSPACES= -e WORKER_ENVIRONMENT=DOCKER -e AIRBYTE_ROLE= -e APPLY_FIELD_SELECTION=false -e WORKER_JOB_ATTEMPT=0 -e OTEL_COLLECTOR_ENDPOINT=http://host.docker.internal:4317 -e FEATURE_FLAG_CLIENT=config -e AIRBYTE_VERSION=0.50.5 -e WORKER_JOB_ID=16 airbyte/source-s3:3.1.0 read --config source_config.json --catalog source_catalog.json
2023-07-20 01:18:30 INFO i.a.w.i.VersionedAirbyteStreamFactory(create):177 - Reading messages from protocol version 0.2.0
2023-07-20 01:18:30 INFO i.a.w.i.HeartbeatTimeoutChaperone(runWithHeartbeatThread):94 - Starting source heartbeat check. Will check every 1 minutes.
2023-07-20 01:18:30 INFO i.a.w.g.DefaultReplicationWorker(lambda$readFromDstRunnable$4):228 - Destination output thread started.
2023-07-20 01:18:30 INFO i.a.w.g.DefaultReplicationWorker(lambda$readFromSrcAndWriteToDstRunnable$5):272 - Replication thread started.
2023-07-20 01:18:32 source > initialised stream with format: 
2023-07-20 01:18:32 source > Starting syncing SourceS3
2023-07-20 01:18:32 source > initialised stream with format: {'encoding': 'utf8', 'filetype': 'csv', 'delimiter': ',', 'block_size': 10000, 'quote_char': '"', 'double_quote': True, 'infer_datatypes': True, 'newlines_in_values': False}
2023-07-20 01:18:32 source > Marking stream chicago_food_inspections as STARTED
2023-07-20 01:18:32 destination > INFO i.a.i.d.c.ClickhouseDestination(main):113 starting destination: class io.airbyte.integrations.destination.clickhouse.ClickhouseDestination
2023-07-20 01:18:32 destination > INFO i.a.i.b.IntegrationCliParser(parseOptions):126 integration args: {catalog=destination_catalog.json, write=null, config=destination_config.json}
2023-07-20 01:18:32 destination > INFO i.a.i.b.IntegrationRunner(runInternal):106 Running integration: io.airbyte.integrations.base.ssh.SshWrappedDestination
2023-07-20 01:18:32 destination > INFO i.a.i.b.IntegrationRunner(runInternal):107 Command: WRITE
2023-07-20 01:18:32 destination > INFO i.a.i.b.IntegrationRunner(runInternal):108 Integration config: IntegrationConfig{command=WRITE, configPath='destination_config.json', catalogPath='destination_catalog.json', statePath='null'}
2023-07-20 01:18:32 destination > WARN c.n.s.JsonMetaSchema(newValidator):278 Unknown keyword order - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2023-07-20 01:18:32 destination > WARN c.n.s.JsonMetaSchema(newValidator):278 Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2023-07-20 01:18:32 destination > INFO i.a.i.b.s.SshTunnel(getInstance):204 Starting connection with method: NO_TUNNEL
2023-07-20 01:18:32 destination > INFO c.z.h.HikariDataSource(<init>):80 HikariPool-1 - Starting...
2023-07-20 01:18:32 destination > INFO c.z.h.HikariDataSource(<init>):82 HikariPool-1 - Start completed.
2023-07-20 01:18:33 destination > INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$toWriteConfig$0):103 Write config: WriteConfig{streamName=chicago_food_inspections, namespace=null, outputSchemaName=db1, tmpTableName=_airbyte_tmp_qik_chicago_food_inspections, outputTableName=_airbyte_raw_chicago_food_inspections, syncMode=overwrite}
2023-07-20 01:18:33 destination > INFO i.a.i.d.b.BufferedStreamConsumer(startTracked):144 class io.airbyte.integrations.destination.buffered_stream_consumer.BufferedStreamConsumer started.
2023-07-20 01:18:33 destination > INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onStartFunction$1):142 Preparing raw tables in destination started for 1 streams
2023-07-20 01:18:33 destination > INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onStartFunction$1):147 Preparing raw table in destination started for stream chicago_food_inspections. schema: db1, table name: _airbyte_raw_chicago_food_inspections
2023-07-20 01:18:33 destination > INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onStartFunction$1):160 Preparing raw tables in destination completed.
2023-07-20 01:18:34 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 5544 records (24 MB)
2023-07-20 01:18:34 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 5544
2023-07-20 01:18:35 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:18:36 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 4577 records (24 MB)
2023-07-20 01:18:36 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 4577
2023-07-20 01:18:36 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:18:38 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 4681 records (24 MB)
2023-07-20 01:18:38 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 4681
2023-07-20 01:18:38 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:18:40 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 4722 records (24 MB)
2023-07-20 01:18:40 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 4722
2023-07-20 01:18:40 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:18:42 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 5006 records (24 MB)
2023-07-20 01:18:42 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 5006
2023-07-20 01:18:42 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:18:43 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 4526 records (24 MB)
2023-07-20 01:18:43 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 4526
2023-07-20 01:18:44 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:18:45 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 4984 records (24 MB)
2023-07-20 01:18:45 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 4984
2023-07-20 01:18:46 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:18:47 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 5000 records (24 MB)
2023-07-20 01:18:47 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 5000
2023-07-20 01:18:47 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:18:49 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 4653 records (24 MB)
2023-07-20 01:18:49 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 4653
2023-07-20 01:18:49 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:18:51 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 4769 records (24 MB)
2023-07-20 01:18:51 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 4769
2023-07-20 01:18:51 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:18:53 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 4898 records (24 MB)
2023-07-20 01:18:53 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 4898
2023-07-20 01:18:53 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:18:55 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 4146 records (24 MB)
2023-07-20 01:18:55 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 4146
2023-07-20 01:18:55 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:18:56 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 4090 records (24 MB)
2023-07-20 01:18:56 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 4090
2023-07-20 01:18:56 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:18:58 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 4195 records (24 MB)
2023-07-20 01:18:58 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 4195
2023-07-20 01:18:58 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:18:59 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 3977 records (24 MB)
2023-07-20 01:18:59 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 3977
2023-07-20 01:19:00 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:19:01 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 3650 records (24 MB)
2023-07-20 01:19:01 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 3650
2023-07-20 01:19:01 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:19:02 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 3680 records (24 MB)
2023-07-20 01:19:02 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 3680
2023-07-20 01:19:03 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:19:04 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 4134 records (24 MB)
2023-07-20 01:19:04 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 4134
2023-07-20 01:19:04 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:19:06 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 3585 records (24 MB)
2023-07-20 01:19:06 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 3585
2023-07-20 01:19:06 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:19:07 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 3778 records (24 MB)
2023-07-20 01:19:07 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 3778
2023-07-20 01:19:07 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:19:09 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 4483 records (24 MB)
2023-07-20 01:19:09 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 4483
2023-07-20 01:19:09 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:19:11 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 4473 records (24 MB)
2023-07-20 01:19:11 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 4473
2023-07-20 01:19:11 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:19:13 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 4766 records (24 MB)
2023-07-20 01:19:13 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 4766
2023-07-20 01:19:13 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:19:14 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 4744 records (24 MB)
2023-07-20 01:19:14 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 4744
2023-07-20 01:19:14 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:19:16 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 4965 records (24 MB)
2023-07-20 01:19:16 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 4965
2023-07-20 01:19:16 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:19:18 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 4593 records (24 MB)
2023-07-20 01:19:18 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 4593
2023-07-20 01:19:18 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:19:20 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 4833 records (24 MB)
2023-07-20 01:19:20 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 4833
2023-07-20 01:19:20 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:19:22 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 4966 records (24 MB)
2023-07-20 01:19:22 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 4966
2023-07-20 01:19:22 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:19:23 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 4622 records (24 MB)
2023-07-20 01:19:23 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 4622
2023-07-20 01:19:24 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:19:25 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 4207 records (24 MB)
2023-07-20 01:19:25 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 4207
2023-07-20 01:19:25 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:19:27 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 4736 records (24 MB)
2023-07-20 01:19:27 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 4736
2023-07-20 01:19:27 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:19:29 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 5055 records (24 MB)
2023-07-20 01:19:29 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 5055
2023-07-20 01:19:29 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:19:31 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 4908 records (24 MB)
2023-07-20 01:19:31 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 4908
2023-07-20 01:19:31 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:19:33 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 4183 records (24 MB)
2023-07-20 01:19:33 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 4183
2023-07-20 01:19:33 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:19:35 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 4435 records (24 MB)
2023-07-20 01:19:35 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 4435
2023-07-20 01:19:35 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:19:36 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 4771 records (24 MB)
2023-07-20 01:19:36 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 4771
2023-07-20 01:19:37 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:19:38 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 4943 records (24 MB)
2023-07-20 01:19:38 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 4943
2023-07-20 01:19:38 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:19:40 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 4444 records (24 MB)
2023-07-20 01:19:40 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 4444
2023-07-20 01:19:40 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:19:42 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 4715 records (24 MB)
2023-07-20 01:19:42 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 4715
2023-07-20 01:19:42 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:19:44 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 4904 records (24 MB)
2023-07-20 01:19:44 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 4904
2023-07-20 01:19:44 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:19:46 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 5765 records (24 MB)
2023-07-20 01:19:46 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 5765
2023-07-20 01:19:46 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:19:48 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 4716 records (24 MB)
2023-07-20 01:19:48 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 4716
2023-07-20 01:19:48 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:19:50 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 5022 records (24 MB)
2023-07-20 01:19:50 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 5022
2023-07-20 01:19:50 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:19:52 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 5211 records (24 MB)
2023-07-20 01:19:52 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 5211
2023-07-20 01:19:52 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:19:54 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 4984 records (24 MB)
2023-07-20 01:19:54 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 4984
2023-07-20 01:19:54 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:19:55 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 4881 records (24 MB)
2023-07-20 01:19:55 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 4881
2023-07-20 01:19:56 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:19:57 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 4220 records (24 MB)
2023-07-20 01:19:57 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 4220
2023-07-20 01:19:57 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:19:59 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 3787 records (24 MB)
2023-07-20 01:19:59 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 3787
2023-07-20 01:19:59 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:20:00 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 4042 records (24 MB)
2023-07-20 01:20:00 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 4042
2023-07-20 01:20:00 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:30:19 destination > INFO i.a.i.b.FailureTrackingAirbyteMessageConsumer(close):80 Airbyte message consumer: succeeded.
2023-07-20 01:30:19 destination > INFO i.a.i.d.b.BufferedStreamConsumer(close):255 executing on success close procedure.
2023-07-20 01:30:19 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing chicago_food_inspections: 1843 records (12 MB)
2023-07-20 01:30:19 destination > INFO i.a.i.d.c.ClickhouseSqlOperations(insertRecordsInternal):67 actual size of batch: 1843
2023-07-20 01:30:19 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for chicago_food_inspections
2023-07-20 01:30:19 destination > INFO i.a.i.b.IntegrationRunner(runInternal):195 Completed integration: io.airbyte.integrations.base.ssh.SshWrappedDestination
2023-07-20 01:30:19 destination > INFO i.a.i.d.c.ClickhouseDestination(main):115 completed destination: class io.airbyte.integrations.destination.clickhouse.ClickhouseDestination
2023-07-20 01:30:20 ERROR i.a.w.g.DefaultReplicationWorker(replicate):215 - Sync worker failed.
io.airbyte.workers.internal.exception.SourceException: Source process exited with non-zero exit code 1
    at io.airbyte.workers.general.DefaultReplicationWorker.lambda$readFromSrcAndWriteToDstRunnable$5(DefaultReplicationWorker.java:315) ~[io.airbyte-airbyte-commons-worker-0.50.5.jar:?]
    at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804) ~[?:?]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
    at java.lang.Thread.run(Thread.java:1589) ~[?:?]
    Suppressed: io.airbyte.workers.exception.WorkerException: Source process exit with code 1. This warning is normal if the job was cancelled.
        at io.airbyte.workers.internal.DefaultAirbyteSource.close(DefaultAirbyteSource.java:145) ~[io.airbyte-airbyte-commons-worker-0.50.5.jar:?]
        at io.airbyte.workers.general.DefaultReplicationWorker.replicate(DefaultReplicationWorker.java:163) ~[io.airbyte-airbyte-commons-worker-0.50.5.jar:?]
        at io.airbyte.workers.general.DefaultReplicationWorker.run(DefaultReplicationWorker.java:147) ~[io.airbyte-airbyte-commons-worker-0.50.5.jar:?]
        at io.airbyte.workers.general.DefaultReplicationWorker.run(DefaultReplicationWorker.java:62) ~[io.airbyte-airbyte-commons-worker-0.50.5.jar:?]
        at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$5(TemporalAttemptExecution.java:195) ~[io.airbyte-airbyte-workers-0.50.5.jar:?]
        at java.lang.Thread.run(Thread.java:1589) ~[?:?]
2023-07-20 01:30:20 INFO i.a.w.g.ReplicationWorkerHelper(getReplicationOutput):288 - sync summary: {
  "status" : "failed",
  "recordsSynced" : 226812,
  "bytesSynced" : 323572078,
  "startTime" : 1689815910676,
  "endTime" : 1689816620161,
  "totalStats" : {
    "bytesCommitted" : 0,
    "bytesEmitted" : 323572078,
    "destinationStateMessagesEmitted" : 0,
    "destinationWriteEndTime" : 1689816620160,
    "destinationWriteStartTime" : 1689815910676,
    "meanSecondsBeforeSourceStateMessageEmitted" : 0,
    "maxSecondsBeforeSourceStateMessageEmitted" : 0,
    "maxSecondsBetweenStateMessageEmittedandCommitted" : 0,
    "meanSecondsBetweenStateMessageEmittedandCommitted" : 0,
    "recordsEmitted" : 226812,
    "recordsCommitted" : 0,
    "replicationEndTime" : 0,
    "replicationStartTime" : 1689815910676,
    "sourceReadEndTime" : 1689816619411,
    "sourceReadStartTime" : 1689815910717,
    "sourceStateMessagesEmitted" : 0
  },
  "streamStats" : [ {
    "streamName" : "chicago_food_inspections",
    "stats" : {
      "bytesEmitted" : 323572078,
      "recordsEmitted" : 226812
    }
  } ]
}
2023-07-20 01:30:20 INFO i.a.w.g.ReplicationWorkerHelper(getReplicationOutput):289 - failures: [ {
  "failureOrigin" : "source",
  "failureType" : "config_error",
  "externalMessage" : "Possibly too small block size used. Please try to increase it",
  "metadata" : {
    "attemptNumber" : 0,
    "jobId" : 16,
    "from_trace_message" : true,
    "connector_command" : "read"
  },
  "stacktrace" : "Traceback (most recent call last):\n  File \"/airbyte/integration_code/source_s3/source_files_abstract/formats/csv_parser.py\", line 245, in stream_records\n    batch = streaming_reader.read_next_batch()\n  File \"pyarrow/ipc.pxi\", line 683, in pyarrow.lib.RecordBatchReader.read_next_batch\n  File \"pyarrow/error.pxi\", line 100, in pyarrow.lib.check_status\npyarrow.lib.ArrowInvalid: straddling object straddles two block boundaries (try to increase block size?)\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n  File \"/airbyte/integration_code/main.py\", line 13, in <module>\n    launch(source, sys.argv[1:])\n  File \"/usr/local/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py\", line 171, in launch\n    for message in source_entrypoint.run(parsed_args):\n  File \"/usr/local/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py\", line 101, in run\n    yield from map(AirbyteEntrypoint.airbyte_message_to_string, self.read(source_spec, config, config_catalog, state))\n  File \"/usr/local/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py\", line 141, in read\n    yield from self.source.read(self.logger, config, catalog, state)\n  File \"/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/abstract_source.py\", line 132, in read\n    raise e\n  File \"/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/abstract_source.py\", line 121, in read\n    yield from self._read_stream(\n  File \"/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/abstract_source.py\", line 196, in _read_stream\n    for record in record_iterator:\n  File \"/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/abstract_source.py\", line 327, in _read_full_refresh\n    for record_data_or_message in record_data_or_messages:\n  File \"/airbyte/integration_code/source_s3/source_files_abstract/stream.py\", line 300, in read_records\n    yield from self._read_from_slice(file_reader, stream_slice)\n  File \"/airbyte/integration_code/source_s3/source_files_abstract/stream.py\", line 274, in _read_from_slice\n    for record in file_reader.stream_records(f, storage_file.file_info):\n  File \"/airbyte/integration_code/source_s3/source_files_abstract/formats/csv_parser.py\", line 248, in stream_records\n    raise AirbyteTracedException(message=error_message, failure_type=FailureType.config_error) from e\nairbyte_cdk.utils.traced_exception.AirbyteTracedException: None\n",
  "timestamp" : 1689816001392
}, {
  "failureOrigin" : "source",
  "internalMessage" : "Source process exited with non-zero exit code 1",
  "externalMessage" : "Something went wrong within the source connector",
  "metadata" : {
    "attemptNumber" : 0,
    "jobId" : 16,
    "connector_command" : "read"
  },
  "stacktrace" : "io.airbyte.workers.internal.exception.SourceException: Source process exited with non-zero exit code 1\n\tat io.airbyte.workers.general.DefaultReplicationWorker.lambda$readFromSrcAndWriteToDstRunnable$5(DefaultReplicationWorker.java:315)\n\tat java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)\n\tat java.base/java.lang.Thread.run(Thread.java:1589)\n",
  "timestamp" : 1689816619411
} ]
2023-07-20 01:30:20 INFO i.a.w.t.TemporalAttemptExecution(get):163 - Stopping cancellation check scheduling...
2023-07-20 01:30:20 INFO i.a.c.i.LineGobbler(voidCall):149 - 
2023-07-20 01:30:20 INFO i.a.c.i.LineGobbler(voidCall):149 - ----- END REPLICATION -----
2023-07-20 01:30:20 INFO i.a.c.i.LineGobbler(voidCall):149 - 
2023-07-20 01:30:20 INFO i.a.w.t.s.ReplicationActivityImpl(lambda$replicate$3):161 - sync summary: io.airbyte.config.StandardSyncOutput@6802067a[standardSyncSummary=io.airbyte.config.StandardSyncSummary@3736c709[status=failed,recordsSynced=226812,bytesSynced=323572078,startTime=1689815910676,endTime=1689816620161,totalStats=io.airbyte.config.SyncStats@6e44b604[bytesCommitted=0,bytesEmitted=323572078,destinationStateMessagesEmitted=0,destinationWriteEndTime=1689816620160,destinationWriteStartTime=1689815910676,estimatedBytes=<null>,estimatedRecords=<null>,meanSecondsBeforeSourceStateMessageEmitted=0,maxSecondsBeforeSourceStateMessageEmitted=0,maxSecondsBetweenStateMessageEmittedandCommitted=0,meanSecondsBetweenStateMessageEmittedandCommitted=0,recordsEmitted=226812,recordsCommitted=0,replicationEndTime=0,replicationStartTime=1689815910676,sourceReadEndTime=1689816619411,sourceReadStartTime=1689815910717,sourceStateMessagesEmitted=0,additionalProperties={}],streamStats=[io.airbyte.config.StreamSyncStats@1c07c3a5[streamName=chicago_food_inspections,streamNamespace=<null>,stats=io.airbyte.config.SyncStats@64ce2789[bytesCommitted=<null>,bytesEmitted=323572078,destinationStateMessagesEmitted=<null>,destinationWriteEndTime=<null>,destinationWriteStartTime=<null>,estimatedBytes=<null>,estimatedRecords=<null>,meanSecondsBeforeSourceStateMessageEmitted=<null>,maxSecondsBeforeSourceStateMessageEmitted=<null>,maxSecondsBetweenStateMessageEmittedandCommitted=<null>,meanSecondsBetweenStateMessageEmittedandCommitted=<null>,recordsEmitted=226812,recordsCommitted=<null>,replicationEndTime=<null>,replicationStartTime=<null>,sourceReadEndTime=<null>,sourceReadStartTime=<null>,sourceStateMessagesEmitted=<null>,additionalProperties={}],additionalProperties={}]],additionalProperties={}],normalizationSummary=<null>,webhookOperationSummary=<null>,state=<null>,outputCatalog=io.airbyte.protocol.models.ConfiguredAirbyteCatalog@6541ba96[streams=[io.airbyte.protocol.models.ConfiguredAirbyteStream@2efed4ac[stream=io.airbyte.protocol.models.AirbyteStream@cc6203d[name=chicago_food_inspections,jsonSchema={"type":"object","properties":{"Zip":{"type":["null","integer"]},"Inspection Type":{"type":["null","string"]},"Address":{"type":["null","string"]},"AKA Name":{"type":["null","string"]},"Inspection Date":{"type":["null","string"]},"Risk":{"type":["null","string"]},"_ab_source_file_last_modified":{"format":"date-time","type":"string"},"Latitude":{"type":["null","number"]},"City":{"type":["null","string"]},"Longitude":{"type":["null","number"]},"_ab_source_file_url":{"type":"string"},"Facility Type":{"type":["null","string"]},"Results":{"type":["null","string"]},"Violations":{"type":["null","string"]},"Inspection ID":{"type":["null","integer"]},"State":{"type":["null","string"]},"DBA Name":{"type":["null","string"]},"License #":{"type":["null","integer"]},"Location":{"type":["null","string"]}}},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=true,defaultCursorField=[_ab_source_file_last_modified],sourceDefinedPrimaryKey=[],namespace=<null>,additionalProperties={}],syncMode=full_refresh,cursorField=[_ab_source_file_last_modified],destinationSyncMode=overwrite,primaryKey=[],additionalProperties={}]],additionalProperties={}],failures=[io.airbyte.config.FailureReason@19a1bc43[failureOrigin=source,failureType=config_error,internalMessage=<null>,externalMessage=Possibly too small block size used. Please try to increase it,metadata=io.airbyte.config.Metadata@715c30eb[additionalProperties={attemptNumber=0, jobId=16, from_trace_message=true, connector_command=read}],stacktrace=Traceback (most recent call last):
  File "/airbyte/integration_code/source_s3/source_files_abstract/formats/csv_parser.py", line 245, in stream_records
    batch = streaming_reader.read_next_batch()
  File "pyarrow/ipc.pxi", line 683, in pyarrow.lib.RecordBatchReader.read_next_batch
  File "pyarrow/error.pxi", line 100, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: straddling object straddles two block boundaries (try to increase block size?)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/airbyte/integration_code/main.py", line 13, in <module>
    launch(source, sys.argv[1:])
  File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py", line 171, in launch
    for message in source_entrypoint.run(parsed_args):
  File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py", line 101, in run
    yield from map(AirbyteEntrypoint.airbyte_message_to_string, self.read(source_spec, config, config_catalog, state))
  File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py", line 141, in read
    yield from self.source.read(self.logger, config, catalog, state)
  File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/abstract_source.py", line 132, in read
    raise e
  File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/abstract_source.py", line 121, in read
    yield from self._read_stream(
  File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/abstract_source.py", line 196, in _read_stream
    for record in record_iterator:
  File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/abstract_source.py", line 327, in _read_full_refresh
    for record_data_or_message in record_data_or_messages:
  File "/airbyte/integration_code/source_s3/source_files_abstract/stream.py", line 300, in read_records
    yield from self._read_from_slice(file_reader, stream_slice)
  File "/airbyte/integration_code/source_s3/source_files_abstract/stream.py", line 274, in _read_from_slice
    for record in file_reader.stream_records(f, storage_file.file_info):
  File "/airbyte/integration_code/source_s3/source_files_abstract/formats/csv_parser.py", line 248, in stream_records
    raise AirbyteTracedException(message=error_message, failure_type=FailureType.config_error) from e
airbyte_cdk.utils.traced_exception.AirbyteTracedException: None
,retryable=<null>,timestamp=1689816001392,additionalProperties={}], io.airbyte.config.FailureReason@29dfc1d8[failureOrigin=source,failureType=<null>,internalMessage=Source process exited with non-zero exit code 1,externalMessage=Something went wrong within the source connector,metadata=io.airbyte.config.Metadata@1425cb16[additionalProperties={attemptNumber=0, jobId=16, connector_command=read}],stacktrace=io.airbyte.workers.internal.exception.SourceException: Source process exited with non-zero exit code 1
    at io.airbyte.workers.general.DefaultReplicationWorker.lambda$readFromSrcAndWriteToDstRunnable$5(DefaultReplicationWorker.java:315)
    at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
    at java.base/java.lang.Thread.run(Thread.java:1589)
,retryable=<null>,timestamp=1689816619411,additionalProperties={}]],commitStateAsap=true,additionalProperties={}]
2023-07-20 01:30:20 INFO i.a.w.t.s.ReplicationActivityImpl(lambda$replicate$3):166 - Sync summary length: 7014
2023-07-20 01:30:20 INFO i.a.c.t.TemporalUtils(withBackgroundHeartbeat):307 - Stopping temporal heartbeating...
2023-07-20 01:30:20 INFO i.a.w.t.s.NormalizationActivityImpl(lambda$normalize$3):128 - Using protocol v0
2023-07-20 01:30:20 INFO i.a.w.t.s.NormalizationActivityImpl(lambda$normalize$3):148 - Using normalization: airbyte/normalization-clickhouse:0.4.3
2023-07-20 01:30:20 INFO i.a.w.t.TemporalAttemptExecution(get):136 - Docker volume job log path: /tmp/workspace/16/0/logs.log
2023-07-20 01:30:20 INFO i.a.w.t.TemporalAttemptExecution(get):141 - Executing worker wrapper. Airbyte version: 0.50.5
2023-07-20 01:30:20 INFO i.a.a.c.AirbyteApiClient(retryWithJitterThrows):229 - Attempt 0 to save workflow id for cancellation
2023-07-20 01:30:20 INFO i.a.c.i.LineGobbler(voidCall):149 - 
2023-07-20 01:30:20 INFO i.a.w.n.DefaultNormalizationRunner(runProcess):130 - Running with normalization version: airbyte/normalization-clickhouse:0.4.3
2023-07-20 01:30:20 INFO i.a.c.i.LineGobbler(voidCall):149 - ----- START DEFAULT NORMALIZATION -----
2023-07-20 01:30:20 INFO i.a.c.i.LineGobbler(voidCall):149 - 
2023-07-20 01:30:20 INFO i.a.c.i.LineGobbler(voidCall):149 - Checking if airbyte/normalization-clickhouse:0.4.3 exists...
2023-07-20 01:30:20 INFO i.a.c.i.LineGobbler(voidCall):149 - airbyte/normalization-clickhouse:0.4.3 was found locally.
2023-07-20 01:30:20 INFO i.a.w.p.DockerProcessFactory(create):139 - Creating docker container = normalization-clickhouse-normalize-16-0-zzxjg with resources io.airbyte.config.ResourceRequirements@436a5e29[cpuRequest=,cpuLimit=,memoryRequest=,memoryLimit=,additionalProperties={}] and allowedHosts null
2023-07-20 01:30:20 INFO i.a.w.p.DockerProcessFactory(create):192 - Preparing command: docker run --rm --init -i -w /data/16/0/normalize --log-driver none --name normalization-clickhouse-normalize-16-0-zzxjg --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e WORKER_ENVIRONMENT=DOCKER -e AIRBYTE_ROLE= -e AIRBYTE_VERSION=0.50.5 airbyte/normalization-clickhouse:0.4.3 run --integration-type clickhouse --config destination_config.json --catalog destination_catalog.json
2023-07-20 01:30:20 normalization > Running: transform-config --config destination_config.json --integration-type clickhouse --out /data/16/0/normalize
2023-07-20 01:30:21 normalization > Namespace(config='destination_config.json', integration_type=<DestinationType.CLICKHOUSE: 'clickhouse'>, out='/data/16/0/normalize')
2023-07-20 01:30:21 normalization > transform_clickhouse
2023-07-20 01:30:21 normalization > Running: transform-catalog --integration-type clickhouse --profile-config-dir /data/16/0/normalize --catalog destination_catalog.json --out /data/16/0/normalize/models/generated/ --json-column _airbyte_data
2023-07-20 01:30:21 normalization > Processing destination_catalog.json...
2023-07-20 01:30:21 normalization >   Generating airbyte_ctes/db1/chicago_food_inspections_ab1.sql from chicago_food_inspections
2023-07-20 01:30:21 normalization >   Generating airbyte_ctes/db1/chicago_food_inspections_ab2.sql from chicago_food_inspections
2023-07-20 01:30:21 normalization >   Generating airbyte_ctes/db1/chicago_food_inspections_ab3.sql from chicago_food_inspections
2023-07-20 01:30:21 normalization >   Adding drop table hook for chicago_food_inspections_scd to chicago_food_inspections
2023-07-20 01:30:21 normalization >   Generating airbyte_tables/db1/chicago_food_inspections.sql from chicago_food_inspections
2023-07-20 01:30:22 normalization > detected no config file for ssh, assuming ssh is off.
2023-07-20 01:30:27 normalization > 01:30:27  Running with dbt=1.4.6
2023-07-20 01:30:27 normalization > 01:30:27  Unable to do partial parsing because saved manifest not found. Starting full parse.
2023-07-20 01:30:29 normalization > 01:30:29  [WARNING]: Configuration paths exist in your dbt_project.yml file which do not apply to any resources.
2023-07-20 01:30:29 normalization > There are 2 unused configuration paths:
2023-07-20 01:30:29 normalization > - models.airbyte_utils.generated.airbyte_views
2023-07-20 01:30:29 normalization > - models.airbyte_utils.generated.airbyte_incremental
2023-07-20 01:30:29 normalization > 01:30:29  Found 4 models, 0 tests, 0 snapshots, 0 analyses, 753 macros, 0 operations, 0 seed files, 1 source, 0 exposures, 0 metrics
2023-07-20 01:30:29 normalization > 01:30:29  
2023-07-20 01:30:29 normalization > 01:30:29  Concurrency: 1 threads (target='prod')
2023-07-20 01:30:29 normalization > 01:30:29  
2023-07-20 01:30:29 normalization > 01:30:29  1 of 1 START sql table model db1.chicago_food_inspections .............................................................. [RUN]
2023-07-20 01:30:35 normalization > 01:30:35  1 of 1 OK created sql table model db1.chicago_food_inspections ......................................................... [OK in 5.60s]
2023-07-20 01:30:35 normalization > 01:30:35  
2023-07-20 01:30:35 normalization > 01:30:35  Finished running 1 table model in 0 hours 0 minutes and 5.89 seconds (5.89s).
2023-07-20 01:30:35 normalization > 01:30:35  
2023-07-20 01:30:35 normalization > 01:30:35  Completed successfully
2023-07-20 01:30:35 normalization > 01:30:35  
2023-07-20 01:30:35 normalization > 01:30:35  Done. PASS=1 WARN=0 ERROR=0 SKIP=0 TOTAL=1
2023-07-20 01:30:36 INFO i.a.w.n.DefaultNormalizationRunner(close):196 - Terminating normalization process...
2023-07-20 01:30:36 normalization > Unable to connect ClickHouse Connect C to Numpy API [No module named 'numpy'], falling back to pure Python
2023-07-20 01:30:36 INFO i.a.w.n.DefaultNormalizationRunner(close):209 - Normalization process successfully terminated.
2023-07-20 01:30:36 INFO i.a.w.g.DefaultNormalizationWorker(run):97 - Normalization executed in 16 seconds for job 16.
2023-07-20 01:30:36 INFO i.a.w.g.DefaultNormalizationWorker(run):109 - Normalization summary: io.airbyte.config.NormalizationSummary@63d48c19[startTime=1689816620282,endTime=1689816636366,failures=[],additionalProperties={}]
2023-07-20 01:30:36 INFO i.a.w.t.TemporalAttemptExecution(get):163 - Stopping cancellation check scheduling...
2023-07-20 01:30:36 INFO i.a.c.i.LineGobbler(voidCall):149 - 
2023-07-20 01:30:36 INFO i.a.c.t.TemporalUtils(withBackgroundHeartbeat):307 - Stopping temporal heartbeating...
2023-07-20 01:30:36 INFO i.a.c.i.LineGobbler(voidCall):149 - ----- END DEFAULT NORMALIZATION -----
2023-07-20 01:30:36 INFO i.a.c.i.LineGobbler(voidCall):149 -
roman-yermilov-gl commented 12 months ago

Closed due to new version using file-based CDK release