airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
16.07k stars 4.11k forks source link

[destination-mongodb] failing to connect #29107

Open agaddis02 opened 1 year ago

agaddis02 commented 1 year ago

Connector Name

destination-mongodb

Connector Version

0.2.0

What step the error happened?

Configuring a new connector

Revelant information

as a level set, I created a mongdb Source, and works perfectly fine, I enter the same information for a destination, and continue to get an error.

I am not using any ssh tunnel configuration, So there is no difference between the configuration for the source mongo and the destination mongo

Log file below check_connection_destination-failure-0d5262ef-5385-43a9-aa81-5a2f545bcd64.txt

Relevant log output

2023-08-04 20:36:23 INFO i.a.w.t.TemporalAttemptExecution(get):136 - Docker volume job log path: /tmp/workspace/0d5262ef-5385-43a9-aa81-5a2f545bcd64/0/logs.log
2023-08-04 20:36:23 INFO i.a.w.t.TemporalAttemptExecution(get):141 - Executing worker wrapper. Airbyte version: 0.50.13
2023-08-04 20:36:23 INFO i.a.a.c.AirbyteApiClient(retryWithJitterThrows):237 - Attempt 0 to save workflow id for cancellation
2023-08-04 20:36:23 INFO i.a.c.EnvConfigs(getEnvOrDefault):1222 - Using default value for environment variable SIDECAR_KUBE_CPU_LIMIT: '2.0'
2023-08-04 20:36:23 INFO i.a.c.EnvConfigs(getEnvOrDefault):1222 - Using default value for environment variable SOCAT_KUBE_CPU_LIMIT: '2.0'
2023-08-04 20:36:23 INFO i.a.c.EnvConfigs(getEnvOrDefault):1222 - Using default value for environment variable SIDECAR_KUBE_CPU_REQUEST: '0.1'
2023-08-04 20:36:23 INFO i.a.c.EnvConfigs(getEnvOrDefault):1222 - Using default value for environment variable SOCAT_KUBE_CPU_REQUEST: '0.1'
2023-08-04 20:36:23 INFO i.a.c.EnvConfigs(getEnvOrDefault):1222 - Using default value for environment variable LAUNCHDARKLY_KEY: ''
2023-08-04 20:36:23 INFO i.a.c.i.LineGobbler(voidCall):149 - 
2023-08-04 20:36:23 INFO i.a.c.i.LineGobbler(voidCall):149 - ----- START CHECK -----
2023-08-04 20:36:23 INFO i.a.c.i.LineGobbler(voidCall):149 - 
2023-08-04 20:36:23 INFO i.a.c.i.LineGobbler(voidCall):149 - Checking if airbyte/destination-mongodb:0.2.0 exists...
2023-08-04 20:36:23 INFO i.a.c.i.LineGobbler(voidCall):149 - airbyte/destination-mongodb:0.2.0 was found locally.
2023-08-04 20:36:23 INFO i.a.w.p.DockerProcessFactory(create):143 - Creating docker container = destination-mongodb-check-0d5262ef-5385-43a9-aa81-5a2f545bcd64-0-dnzxc with resources io.airbyte.config.ResourceRequirements@795b9c20[cpuRequest=,cpuLimit=,memoryRequest=,memoryLimit=,additionalProperties={}] and allowedHosts null
2023-08-04 20:36:23 INFO i.a.w.p.DockerProcessFactory(create):196 - Preparing command: docker run --rm --init -i -w /data/0d5262ef-5385-43a9-aa81-5a2f545bcd64/0 --log-driver none --name destination-mongodb-check-0d5262ef-5385-43a9-aa81-5a2f545bcd64-0-dnzxc --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e WORKER_CONNECTOR_IMAGE=airbyte/destination-mongodb:0.2.0 -e AUTO_DETECT_SCHEMA=true -e LAUNCHDARKLY_KEY= -e SOCAT_KUBE_CPU_REQUEST=0.1 -e SOCAT_KUBE_CPU_LIMIT=2.0 -e USE_STREAM_CAPABLE_STATE=true -e FIELD_SELECTION_WORKSPACES= -e AIRBYTE_ROLE= -e WORKER_ENVIRONMENT=DOCKER -e APPLY_FIELD_SELECTION=false -e WORKER_JOB_ATTEMPT=0 -e OTEL_COLLECTOR_ENDPOINT=http://host.docker.internal:4317 -e FEATURE_FLAG_CLIENT=config -e AIRBYTE_VERSION=0.50.13 -e WORKER_JOB_ID=0d5262ef-5385-43a9-aa81-5a2f545bcd64 airbyte/destination-mongodb:0.2.0 check --config source_config.json
2023-08-04 20:36:23 INFO i.a.w.i.VersionedAirbyteStreamFactory(create):177 - Reading messages from protocol version 0.2.0
2023-08-04 20:36:25 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):312 - INFO i.a.i.d.m.MongodbDestination(main):63 starting destination: class io.airbyte.integrations.destination.mongodb.MongodbDestination
2023-08-04 20:36:25 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):312 - INFO i.a.i.b.IntegrationCliParser(parseOptions):126 integration args: {check=null, config=source_config.json}
2023-08-04 20:36:25 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):312 - INFO i.a.i.b.IntegrationRunner(runInternal):106 Running integration: io.airbyte.integrations.base.ssh.SshWrappedDestination
2023-08-04 20:36:25 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):312 - INFO i.a.i.b.IntegrationRunner(runInternal):107 Command: CHECK
2023-08-04 20:36:25 INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):312 - INFO i.a.i.b.IntegrationRunner(runInternal):108 Integration config: IntegrationConfig{command=CHECK, configPath='source_config.json', catalogPath='null', statePath='null'}
2023-08-04 20:36:25 WARN i.a.w.i.VersionedAirbyteStreamFactory(internalLog):309 - WARN c.n.s.JsonMetaSchema(newValidator):278 Unknown keyword order - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword


### Contribute

- [ ] Yes, I want to contribute
agaddis02 commented 1 year ago

Configuration check failed Format specifier '%s'

This is the exact error that is kicked back

jaquim commented 1 year ago

it hangs in this call to String.format. in file airbyte-integrations/connectors/destination-mongodb/src/main/java/io/airbyte/integrations/destination/mongodb/MongodbDestination.java, at line 168

connectionStrBuilder.append( String.format(MongoUtils.MONGODB_SERVER_URL, credentials, instanceConfig.get(JdbcUtils.HOST_KEY).asText(), instanceConfig.get(JdbcUtils.PORT_KEY).asText(), config.get(JdbcUtils.DATABASE_KEY).asText(), tls) );

anyone an idea for a fix?

srinath97 commented 11 months ago

Is there a fix for this issue? I'm stuck too because of "Stack Trace: java.util.MissingFormatArgumentException: Format specifier '%s'"

adymilk commented 10 months ago

Is there a fix for this issue? I'm stuck too because of "Stack Trace: java.util.MissingFormatArgumentException: Format specifier '%s'"

it hangs in this call to String.format. in file airbyte-integrations/connectors/destination-mongodb/src/main/java/io/airbyte/integrations/destination/mongodb/MongodbDestination.java, at line 168

connectionStrBuilder.append( String.format(MongoUtils.MONGODB_SERVER_URL, credentials, instanceConfig.get(JdbcUtils.HOST_KEY).asText(), instanceConfig.get(JdbcUtils.PORT_KEY).asText(), config.get(JdbcUtils.DATABASE_KEY).asText(), tls) );

anyone an idea for a fix?

I have same error, anyone resolved ?

Janzenzzc commented 10 months ago

I found that switching the version of destination to 0.1.9 in settings can temporarily solve the problem.

Janzenzzc commented 10 months ago

And need to use admin user. 4f289510c5d3b9f01d99db39962ea78