StarRocks Airbyte: Discovering schema failed

[alberttwong]

I used StarRocks destination source connector. Tested the connection and it worked. Target was a local CSV. Tested the connection and it worked. When I executed connection, I got an error. Using compiled airbyte connector image atwong/destination-starrocks:latest

Internal message: java.lang.NullPointerException: Cannot invoke "io.airbyte.integrations.base.Source.discover(com.fasterxml.jackson.databind.JsonNode)" because "this.source" is null Failure type: system_error

2023-05-24 21:45:39 INFO i.a.w.t.TemporalAttemptExecution(get):136 - Docker volume job log path: /tmp/workspace/5f9579fc-4019-485b-ae7f-cee6784bfe13/0/logs.log
2023-05-24 21:45:39 INFO i.a.w.t.TemporalAttemptExecution(get):141 - Executing worker wrapper. Airbyte version: 0.44.5
2023-05-24 21:45:39 INFO i.a.a.c.AirbyteApiClient(retryWithJitterThrows):222 - Attempt 0 to save workflow id for cancellation
2023-05-24 21:45:39 INFO i.a.c.EnvConfigs(getEnvOrDefault):1222 - Using default value for environment variable SIDECAR_KUBE_CPU_LIMIT: '2.0'
2023-05-24 21:45:39 INFO i.a.c.EnvConfigs(getEnvOrDefault):1222 - Using default value for environment variable SOCAT_KUBE_CPU_LIMIT: '2.0'
2023-05-24 21:45:39 INFO i.a.c.EnvConfigs(getEnvOrDefault):1222 - Using default value for environment variable SIDECAR_KUBE_CPU_REQUEST: '0.1'
2023-05-24 21:45:39 INFO i.a.c.EnvConfigs(getEnvOrDefault):1222 - Using default value for environment variable SOCAT_KUBE_CPU_REQUEST: '0.1'
2023-05-24 21:45:39 INFO i.a.c.EnvConfigs(getEnvOrDefault):1222 - Using default value for environment variable LAUNCHDARKLY_KEY: ''
2023-05-24 21:45:39 INFO i.a.c.EnvConfigs(getEnvOrDefault):1222 - Using default value for environment variable FEATURE_FLAG_CLIENT: ''
2023-05-24 21:45:39 INFO i.a.c.i.LineGobbler(voidCall):149 - Checking if atwong/destination-starrocks:latest exists...
2023-05-24 21:45:39 INFO i.a.c.i.LineGobbler(voidCall):149 - atwong/destination-starrocks:latest was found locally.
2023-05-24 21:45:39 INFO i.a.w.p.DockerProcessFactory(create):139 - Creating docker container = destination-starrocks-discover-5f9579fc-4019-485b-ae7f-cee6784bfe13-0-vvbcg with resources io.airbyte.config.ResourceRequirements@7c62d14e[cpuRequest=,cpuLimit=,memoryRequest=,memoryLimit=,additionalProperties={}] and allowedHosts null
2023-05-24 21:45:39 INFO i.a.w.p.DockerProcessFactory(create):192 - Preparing command: docker run --rm --init -i -w /data/5f9579fc-4019-485b-ae7f-cee6784bfe13/0 --log-driver none --name destination-starrocks-discover-5f9579fc-4019-485b-ae7f-cee6784bfe13-0-vvbcg --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e WORKER_CONNECTOR_IMAGE=atwong/destination-starrocks:latest -e AUTO_DETECT_SCHEMA=true -e LAUNCHDARKLY_KEY= -e SOCAT_KUBE_CPU_REQUEST=0.1 -e SOCAT_KUBE_CPU_LIMIT=2.0 -e USE_STREAM_CAPABLE_STATE=true -e FIELD_SELECTION_WORKSPACES= -e AIRBYTE_ROLE= -e WORKER_ENVIRONMENT=DOCKER -e APPLY_FIELD_SELECTION=false -e WORKER_JOB_ATTEMPT=0 -e OTEL_COLLECTOR_ENDPOINT=http://host.docker.internal:4317 -e FEATURE_FLAG_CLIENT= -e AIRBYTE_VERSION=0.44.5 -e WORKER_JOB_ID=5f9579fc-4019-485b-ae7f-cee6784bfe13 atwong/destination-starrocks:latest discover --config source_config.json
2023-05-24 21:45:39 INFO i.a.w.i.VersionedAirbyteStreamFactory(create):181 - Reading messages from protocol version 0.2.0
2023-05-24 21:45:40 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):317 - INFO i.a.i.b.IntegrationCliParser(parseOptions):126 integration args: {discover=null, config=source_config.json}
2023-05-24 21:45:40 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):317 - INFO i.a.i.b.IntegrationRunner(runInternal):108 Running integration: io.airbyte.integrations.destination.starrocks.StarRocksDestination
2023-05-24 21:45:40 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):317 - INFO i.a.i.b.IntegrationRunner(runInternal):109 Command: DISCOVER
2023-05-24 21:45:40 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):317 - INFO i.a.i.b.IntegrationRunner(runInternal):110 Integration config: IntegrationConfig{command=DISCOVER, configPath='source_config.json', catalogPath='null', statePath='null'}
2023-05-24 21:45:40 WARN i.a.w.i.VersionedAirbyteStreamFactory(internalLog):314 - WARN c.n.s.JsonMetaSchema(newValidator):278 Unknown keyword order - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2023-05-24 21:45:40 WARN i.a.w.i.VersionedAirbyteStreamFactory(internalLog):314 - WARN c.n.s.JsonMetaSchema(newValidator):278 Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2023-05-24 21:45:41 ERROR i.a.w.i.VersionedAirbyteStreamFactory(internalLog):313 - ERROR i.a.i.b.AirbyteExceptionHandler(uncaughtException):26 Something went wrong in the connector. See the logs for more details. java.lang.NullPointerException: Cannot invoke "io.airbyte.integrations.base.Source.discover(com.fasterxml.jackson.databind.JsonNode)" because "this.source" is null
    at io.airbyte.integrations.base.IntegrationRunner.runInternal(IntegrationRunner.java:132) ~[io.airbyte.airbyte-integrations.bases-base-java-0.44.4.jar:?]

[alberttwong]

[liuzhongjun89]

Currently this is only a destination connector, we can reuse source-mysql as a source connector to extract data from starrocks

[alberttwong]

My mistake.