Source Typeform: Typeform sync stuck on never ending `responses` stream sync

[pablolola]

Connector Name

source-typeform

Connector Version

1.1.0

What step the error happened?

During the sync

Revelant information

TLDR: I'm trying to sync a tiny bit of data from Typeform to a MySQL database, but Airbyte repeatedly syncs the same data over and over again without ever terminating.

My setup

• Source
  • Typeform, connector version 1.1.0
  • I have limited the connector to a single form, and I'm also using a temporal filter to only obtain responses starting on January 1st 2023.
  • For context, the form currently has about ~4,500 responses.
• Target
  • A MySQL v8 database. Nothing specially about this. I have successfully written into this database with other connections in the past.
• Connection
  • I'm syncing the data raw, without normalizing.
  • I'm only syncing the forms and responses streams.
  • For both streams, I'm syncing all fields.
  • For the responses stream I'm using the Incremental mode.

My instance is deployed on an EC2 machine following this documentation: https://docs.airbyte.com/deploying-airbyte/on-aws-ec2

Expected behavior

• Well, simply that the data appears correctly in the target.
• Given the data size, I wouldn't expect the connection to take longer than a few minutes at most, nor a lot of resources to be consumed.

Observed behavior

• The sync seems to never end. The sync job quickly executes the sync of the forms stream (it's just one record), moves on to the responses stream and then syncs continuously the same records over and over and over.
• I have confirmed this because the target table keeps on growing indefinitely and the JSON contents of the synced data keeps accumulating duplicate instances.
• Resource consumption is quite heavy given how tiny this sink is. I initially tried to run this on a t2.medium EC2 host but Airbyte would freeze, apparently from CPU overload. Upgraded to t2.large and similar story. I then upgraded to t2.xlarge and this seems to do the trick. This is happening without any other sync taking place in the same Airbyte instance and no other services being deployed in the EC2 machine.

Relevant log output

2023-10-02 12:50:30 INFO i.a.w.t.TemporalAttemptExecution(get):124 - Docker volume job log path: /tmp/workspace/25/0/logs.log
2023-10-02 12:50:30 INFO i.a.w.t.TemporalAttemptExecution(get):129 - Executing worker wrapper. Airbyte version: 0.50.29
2023-10-02 12:50:30 INFO i.a.a.c.AirbyteApiClient(retryWithJitterThrows):290 - Attempt 0 to save workflow id for cancellation
2023-10-02 12:50:30 INFO i.a.w.g.BufferedReplicationWorker(run):146 - start sync worker. job id: 25 attempt id: 0
2023-10-02 12:50:30 INFO i.a.c.i.LineGobbler(voidCall):149 - 
2023-10-02 12:50:30 INFO i.a.c.i.LineGobbler(voidCall):149 - ----- START REPLICATION -----
2023-10-02 12:50:30 INFO i.a.c.i.LineGobbler(voidCall):149 - 
2023-10-02 12:50:30 INFO i.a.w.i.DefaultAirbyteDestination(start):88 - Running destination...
2023-10-02 12:50:30 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable SIDECAR_KUBE_CPU_LIMIT: '2.0'
2023-10-02 12:50:30 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable SOCAT_KUBE_CPU_LIMIT: '2.0'
2023-10-02 12:50:30 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable SIDECAR_KUBE_CPU_REQUEST: '0.1'
2023-10-02 12:50:30 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable SOCAT_KUBE_CPU_REQUEST: '0.1'
2023-10-02 12:50:30 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable LAUNCHDARKLY_KEY: ''
2023-10-02 12:50:30 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable SIDECAR_KUBE_CPU_LIMIT: '2.0'
2023-10-02 12:50:30 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable SOCAT_KUBE_CPU_LIMIT: '2.0'
2023-10-02 12:50:30 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable SIDECAR_KUBE_CPU_REQUEST: '0.1'
2023-10-02 12:50:30 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable SOCAT_KUBE_CPU_REQUEST: '0.1'
2023-10-02 12:50:30 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable LAUNCHDARKLY_KEY: ''
2023-10-02 12:50:30 INFO i.a.c.i.LineGobbler(voidCall):149 - Checking if airbyte/destination-mysql:0.2.0 exists...
2023-10-02 12:50:30 INFO i.a.c.i.LineGobbler(voidCall):149 - Checking if airbyte/source-typeform:1.1.0 exists...
2023-10-02 12:50:30 INFO i.a.c.i.LineGobbler(voidCall):149 - airbyte/source-typeform:1.1.0 was found locally.
2023-10-02 12:50:30 INFO i.a.c.i.LineGobbler(voidCall):149 - airbyte/destination-mysql:0.2.0 was found locally.
2023-10-02 12:50:30 INFO i.a.w.p.DockerProcessFactory(create):143 - Creating docker container = destination-mysql-write-25-0-oihee with resources io.airbyte.config.ResourceRequirements@350137c6[cpuRequest=0.2,cpuLimit=1,memoryRequest=1Gi,memoryLimit=2Gi,additionalProperties={}] and allowedHosts null
2023-10-02 12:50:30 INFO i.a.w.p.DockerProcessFactory(create):143 - Creating docker container = source-typeform-read-25-0-pnteb with resources io.airbyte.config.ResourceRequirements@ad05034[cpuRequest=0.2,cpuLimit=1,memoryRequest=1Gi,memoryLimit=2Gi,additionalProperties={}] and allowedHosts io.airbyte.config.AllowedHosts@3e32a620[hosts=[api.typeform.com, *.datadoghq.com, *.datadoghq.eu, *.sentry.io],additionalProperties={}]
2023-10-02 12:50:30 INFO i.a.w.p.DockerProcessFactory(create):196 - Preparing command: docker run --rm --init -i -w /data/25/0 --log-driver none --name destination-mysql-write-25-0-oihee --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e WORKER_CONNECTOR_IMAGE=airbyte/destination-mysql:0.2.0 -e AUTO_DETECT_SCHEMA=true -e LAUNCHDARKLY_KEY= -e SOCAT_KUBE_CPU_REQUEST=0.1 -e SOCAT_KUBE_CPU_LIMIT=2.0 -e USE_STREAM_CAPABLE_STATE=true -e FIELD_SELECTION_WORKSPACES= -e WORKER_ENVIRONMENT=DOCKER -e AIRBYTE_ROLE= -e APPLY_FIELD_SELECTION=false -e WORKER_JOB_ATTEMPT=0 -e OTEL_COLLECTOR_ENDPOINT=http://host.docker.internal:4317 -e FEATURE_FLAG_CLIENT=config -e AIRBYTE_VERSION=0.50.29 -e WORKER_JOB_ID=25 --cpus=1 --memory-reservation=1Gi --memory=2Gi airbyte/destination-mysql:0.2.0 write --config destination_config.json --catalog destination_catalog.json
2023-10-02 12:50:30 INFO i.a.w.p.DockerProcessFactory(create):196 - Preparing command: docker run --rm --init -i -w /data/25/0 --log-driver none --name source-typeform-read-25-0-pnteb -e CONCURRENT_SOURCE_STREAM_READ=false --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e WORKER_CONNECTOR_IMAGE=airbyte/source-typeform:1.1.0 -e AUTO_DETECT_SCHEMA=true -e LAUNCHDARKLY_KEY= -e SOCAT_KUBE_CPU_REQUEST=0.1 -e SOCAT_KUBE_CPU_LIMIT=2.0 -e USE_STREAM_CAPABLE_STATE=true -e FIELD_SELECTION_WORKSPACES= -e WORKER_ENVIRONMENT=DOCKER -e AIRBYTE_ROLE= -e APPLY_FIELD_SELECTION=false -e WORKER_JOB_ATTEMPT=0 -e OTEL_COLLECTOR_ENDPOINT=http://host.docker.internal:4317 -e FEATURE_FLAG_CLIENT=config -e AIRBYTE_VERSION=0.50.29 -e WORKER_JOB_ID=25 --cpus=1 --memory-reservation=1Gi --memory=2Gi airbyte/source-typeform:1.1.0 read --config source_config.json --catalog source_catalog.json --state input_state.json
2023-10-02 12:50:30 INFO i.a.w.i.VersionedAirbyteMessageBufferedWriterFactory(createWriter):41 - Writing messages to protocol version 0.2.0
2023-10-02 12:50:30 INFO i.a.w.i.VersionedAirbyteStreamFactory(create):177 - Reading messages from protocol version 0.2.0
2023-10-02 12:50:30 INFO i.a.w.i.VersionedAirbyteStreamFactory(create):177 - Reading messages from protocol version 0.2.0
2023-10-02 12:50:30 INFO i.a.w.g.BufferedReplicationWorker(readFromSource):328 - readFromSource: start
2023-10-02 12:50:30 INFO i.a.w.i.HeartbeatTimeoutChaperone(runWithHeartbeatThread):98 - Starting source heartbeat check. Will check every 1 minutes.
2023-10-02 12:50:30 INFO i.a.w.g.BufferedReplicationWorker(readFromDestination):438 - readFromDestination: start
2023-10-02 12:50:30 INFO i.a.w.g.BufferedReplicationWorker(processMessage):364 - processMessage: start
2023-10-02 12:50:30 INFO i.a.w.g.BufferedReplicationWorker(writeToDestination):405 - writeToDestination: start
2023-10-02 12:50:32 source > Starting syncing SourceTypeform
2023-10-02 12:50:32 source > Marking stream forms as STARTED
2023-10-02 12:50:32 source > Syncing stream: forms 
2023-10-02 12:50:32 source > Marking stream forms as RUNNING
2023-10-02 12:50:32 source > Read 1 records from forms stream
2023-10-02 12:50:32 source > Marking stream forms as STOPPED
2023-10-02 12:50:32 source > Finished syncing forms
2023-10-02 12:50:32 source > SourceTypeform runtimes:
Syncing stream forms 0:00:00.432598
2023-10-02 12:50:32 source > Marking stream responses as STARTED
2023-10-02 12:50:32 source > Syncing stream: responses 
2023-10-02 12:50:33 destination > INFO i.a.i.d.m.MySQLDestination(main):132 starting destination: class io.airbyte.integrations.destination.mysql.MySQLDestination
2023-10-02 12:50:33 destination > INFO i.a.i.b.IntegrationCliParser(parseOptions):126 integration args: {catalog=destination_catalog.json, write=null, config=destination_config.json}
2023-10-02 12:50:33 destination > INFO i.a.i.b.IntegrationRunner(runInternal):106 Running integration: io.airbyte.integrations.base.ssh.SshWrappedDestination
2023-10-02 12:50:33 destination > INFO i.a.i.b.IntegrationRunner(runInternal):107 Command: WRITE
2023-10-02 12:50:33 destination > INFO i.a.i.b.IntegrationRunner(runInternal):108 Integration config: IntegrationConfig{command=WRITE, configPath='destination_config.json', catalogPath='destination_catalog.json', statePath='null'}
2023-10-02 12:50:33 destination > WARN c.n.s.JsonMetaSchema(newValidator):278 Unknown keyword order - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2023-10-02 12:50:33 destination > WARN c.n.s.JsonMetaSchema(newValidator):278 Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2023-10-02 12:50:34 destination > INFO i.a.i.b.s.SshTunnel(getInstance):204 Starting connection with method: NO_TUNNEL
2023-10-02 12:50:34 destination > INFO c.z.h.HikariDataSource(<init>):80 HikariPool-1 - Starting...
2023-10-02 12:50:34 destination > INFO c.z.h.HikariDataSource(<init>):82 HikariPool-1 - Start completed.
2023-10-02 12:50:34 destination > INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$toWriteConfig$0):103 Write config: WriteConfig{streamName=forms, namespace=airbyte_typeform_sync, outputSchemaName=airbyte_typeform_sync, tmpTableName=_airbyte_tmp_cwq_forms, outputTableName=_airbyte_raw_forms, syncMode=overwrite}
2023-10-02 12:50:34 destination > INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$toWriteConfig$0):103 Write config: WriteConfig{streamName=responses, namespace=airbyte_typeform_sync, outputSchemaName=airbyte_typeform_sync, tmpTableName=_airbyte_tmp_ppa_responses, outputTableName=_airbyte_raw_responses, syncMode=append}
2023-10-02 12:50:34 destination > INFO i.a.i.d.b.BufferedStreamConsumer(startTracked):173 class io.airbyte.integrations.destination.buffered_stream_consumer.BufferedStreamConsumer started.
2023-10-02 12:50:34 destination > INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onStartFunction$1):142 Preparing raw tables in destination started for 2 streams
2023-10-02 12:50:34 destination > INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onStartFunction$1):147 Preparing raw table in destination started for stream forms. schema: airbyte_typeform_sync, table name: _airbyte_raw_forms
2023-10-02 12:50:35 source > Marking stream responses as RUNNING
2023-10-02 12:50:35 destination > INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onStartFunction$1):147 Preparing raw table in destination started for stream responses. schema: airbyte_typeform_sync, table name: _airbyte_raw_responses
2023-10-02 12:50:35 destination > INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onStartFunction$1):160 Preparing raw tables in destination completed.
2023-10-02 12:51:05 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5383 records (24 MB)
2023-10-02 12:51:09 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 12:51:09 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing forms: 1 records (24 MB)
2023-10-02 12:51:09 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for forms
2023-10-02 12:51:38 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5382 records (24 MB)
2023-10-02 12:51:40 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 12:52:10 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5383 records (24 MB)
2023-10-02 12:52:11 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 12:52:42 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5388 records (24 MB)
2023-10-02 12:52:42 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 12:53:12 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5398 records (24 MB)
2023-10-02 12:53:12 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 12:53:44 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5377 records (24 MB)
2023-10-02 12:53:44 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 12:54:17 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5386 records (24 MB)
2023-10-02 12:54:18 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 12:54:48 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5379 records (24 MB)
2023-10-02 12:54:49 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 12:55:19 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5391 records (24 MB)
2023-10-02 12:55:20 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 12:55:49 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5395 records (24 MB)
2023-10-02 12:55:50 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 12:56:20 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5379 records (24 MB)
2023-10-02 12:56:21 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 12:56:50 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5388 records (24 MB)
2023-10-02 12:56:50 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 12:57:21 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5383 records (24 MB)
2023-10-02 12:57:22 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 12:57:50 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5393 records (24 MB)
2023-10-02 12:57:51 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 12:58:21 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5383 records (24 MB)
2023-10-02 12:58:22 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 12:58:52 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5383 records (24 MB)
2023-10-02 12:58:53 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 12:59:22 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5387 records (24 MB)
2023-10-02 12:59:23 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 12:59:52 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5396 records (24 MB)
2023-10-02 12:59:53 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:00:30 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5381 records (24 MB)
2023-10-02 13:00:31 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:01:02 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5385 records (24 MB)
2023-10-02 13:01:03 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:01:37 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5380 records (24 MB)
2023-10-02 13:01:38 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:02:07 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5380 records (24 MB)
2023-10-02 13:02:08 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:02:37 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5341 records (24 MB)
2023-10-02 13:02:38 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:03:08 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5345 records (24 MB)
2023-10-02 13:03:09 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:03:41 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5334 records (24 MB)
2023-10-02 13:03:42 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:04:15 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5342 records (24 MB)
2023-10-02 13:04:15 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:04:48 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5348 records (24 MB)
2023-10-02 13:04:48 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:05:22 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5332 records (24 MB)
2023-10-02 13:05:22 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:05:52 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5342 records (24 MB)
2023-10-02 13:05:52 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:06:23 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5349 records (24 MB)
2023-10-02 13:06:24 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:06:56 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5332 records (24 MB)
2023-10-02 13:06:57 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:07:27 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5343 records (24 MB)
2023-10-02 13:07:27 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:07:59 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5348 records (24 MB)
2023-10-02 13:08:00 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:08:32 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5334 records (24 MB)
2023-10-02 13:08:32 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:09:02 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5341 records (24 MB)
2023-10-02 13:09:03 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:09:35 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5347 records (24 MB)
2023-10-02 13:09:36 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:10:05 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5334 records (24 MB)
2023-10-02 13:10:06 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:10:37 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5341 records (24 MB)
2023-10-02 13:10:38 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:11:11 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5347 records (24 MB)
2023-10-02 13:11:12 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:11:43 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5359 records (24 MB)
2023-10-02 13:11:44 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:12:17 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5379 records (24 MB)
2023-10-02 13:12:18 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:12:50 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5379 records (24 MB)
2023-10-02 13:12:51 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:13:20 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5369 records (24 MB)
2023-10-02 13:13:21 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:13:51 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5382 records (24 MB)
2023-10-02 13:13:52 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:14:21 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5376 records (24 MB)
2023-10-02 13:14:22 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:14:51 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5373 records (24 MB)
2023-10-02 13:14:51 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:15:22 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5376 records (24 MB)
2023-10-02 13:15:23 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:15:52 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5376 records (24 MB)
2023-10-02 13:15:53 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:16:24 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5380 records (24 MB)
2023-10-02 13:16:25 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:16:55 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5379 records (24 MB)
2023-10-02 13:16:56 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:17:25 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5368 records (24 MB)
2023-10-02 13:17:26 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:17:56 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5382 records (24 MB)
2023-10-02 13:17:57 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:18:27 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5376 records (24 MB)
2023-10-02 13:18:28 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:18:57 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5373 records (24 MB)
2023-10-02 13:18:58 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:19:30 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5376 records (24 MB)
2023-10-02 13:19:31 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:20:00 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5375 records (24 MB)
2023-10-02 13:20:01 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:20:40 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5381 records (24 MB)
2023-10-02 13:20:42 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:21:13 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5378 records (24 MB)
2023-10-02 13:21:14 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:21:43 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5368 records (24 MB)
2023-10-02 13:21:44 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:22:14 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5381 records (24 MB)
2023-10-02 13:22:15 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:22:44 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5377 records (24 MB)
2023-10-02 13:22:45 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:23:16 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5374 records (24 MB)
2023-10-02 13:23:17 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:23:48 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5375 records (24 MB)
2023-10-02 13:23:48 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:24:21 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5376 records (24 MB)
2023-10-02 13:24:22 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:24:53 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5381 records (24 MB)
2023-10-02 13:24:54 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:25:24 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5387 records (24 MB)
2023-10-02 13:25:25 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:25:54 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5409 records (24 MB)
2023-10-02 13:25:55 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:26:28 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5489 records (24 MB)
2023-10-02 13:26:29 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:26:58 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5510 records (24 MB)
2023-10-02 13:26:59 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:27:30 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5514 records (24 MB)
2023-10-02 13:27:31 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:28:00 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5507 records (24 MB)
2023-10-02 13:28:01 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:28:33 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5516 records (24 MB)
2023-10-02 13:28:33 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:29:06 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5505 records (24 MB)
2023-10-02 13:29:06 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:29:38 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5517 records (24 MB)
2023-10-02 13:29:38 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:30:08 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5505 records (24 MB)
2023-10-02 13:30:09 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:30:42 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5518 records (24 MB)
2023-10-02 13:30:43 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:31:14 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5505 records (24 MB)
2023-10-02 13:31:14 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:31:48 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5518 records (24 MB)
2023-10-02 13:31:48 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:32:19 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5505 records (24 MB)
2023-10-02 13:32:19 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses
2023-10-02 13:32:52 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):85 Flushing responses: 5519 records (24 MB)
2023-10-02 13:32:52 destination > INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(flushAllBuffers):91 Flushing completed for responses

Contribute

• [ ] Yes, I want to contribute

[pablolola]

An update from my side.

The idea of trying different versions of the Typeform source connector came to my mind and I did some tests. I tested the exact same setup that I described initially in this issue, while simply changing the version of the Typeform connector. I tried versions: 0.1.12, 0.2.0, 0.3.0 and 1.0.0. All of them worked perfectly. The synced happened as it should, it was done in a few seconds and the data matches the source.

The moment I switched back to 1.1.0 once more and tried to sync again, I got back to the problems of this issue.

So, according to my own experience, it's clearly the update from 1.0.0 to 1.1.0 that broke this connector. I hope this helps out in fixing it. Thanks.

[pablolola]

Hi,

I just wanted to ask if this issue was tackled in the last couple of releases?

• https://github.com/airbytehq/airbyte/pull/31599
• https://github.com/airbytehq/airbyte/pull/31914

Thanks

[nataliekwong]

@pablolola this may have been resolved with this PR: https://github.com/airbytehq/airbyte/pull/40176

Could you try upgrading to the latest version? If this is still an issue feel free to reopen.