search

airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
16.24k stars 4.15k forks source link

[source-mongodb] SSL error when syncing MongoDb ReplicaSet without SSL/TLS enabled #48431

Open heruscode opened 1 week ago

heruscode commented 1 week ago

Connector Name

source-mongodb

Connector Version

1.5.12

What step the error happened?

During the sync

Relevant information

Airbyte version: 1.1.0 Connector version: 1.5.12

Hi! I am trying to sync a Self-Managed MongoDB Replica Set that do not have SSL/TLS enabled --> BigQuery

The MongoDB cluster is a 3 node ReplicaSet and I am trying to sync from primary node. There is no primary/secondary rotation events happening when I am doing the sync, it always error out.

I see that it is being able to extract the records(it shows "10 loaded" in the connection status) and when I go to BigQuery I also confirm that the records were correctly copied.

Image

But after a few minutes the Sync fails with a SSL/TLS error making the Connection status to be "Failed Sync".

pool-2-thread-1 c.m.i.d.l.SLF4JLogger(info):71 No server chosen by ReadPreferenceServerSelector{readPreference=primary} from cluster description ClusterDescription{type=UNKNOWN, connectionMode=SINGLE, serverDescriptions=[ServerDescription{address=cowboy5.mongo.REDACTED.com:27017, type=UNKNOWN, state=CONNECTING}]}. Waiting for 30000 ms before timing out

Is SSL/TLS still being enforced somehow?

Relevant log output

2024-11-08 20:28:52 source > INFO pool-2-thread-1 c.m.i.d.l.SLF4JLogger(info):71 MongoClient with metadata {"driver": {"name": "mongo-java-driver|sync", "version": "4.11.0"}, "os": {"type": "Linux", "name": "Linux", "architecture": "amd64", "version": "6.1.75+"}, "platform": "Java/Amazon.com Inc./21.0.5+11-LTS"} created with settings MongoClientSettings{readPreference=primary, writeConcern=WriteConcern{w=null, wTimeout=null ms, journal=null}, retryWrites=true, retryReads=true, readConcern=ReadConcern{level=null}, credential=MongoCredential{mechanism=null, userName='REDACTED-airby', source='admin', password=<hidden>, mechanismProperties=<hidden>}, transportSettings=null, streamFactoryFactory=null, commandListeners=[], codecRegistry=ProvidersCodecRegistry{codecProviders=[ValueCodecProvider{}, BsonValueCodecProvider{}, DBRefCodecProvider{}, DBObjectCodecProvider{}, DocumentCodecProvider{}, CollectionCodecProvider{}, IterableCodecProvider{}, MapCodecProvider{}, GeoJsonCodecProvider{}, GridFSFileCodecProvider{}, Jsr310CodecProvider{}, JsonObjectCodecProvider{}, BsonCodecProvider{}, EnumCodecProvider{}, com.mongodb.client.model.mql.ExpressionCodecProvider@67ec8477, com.mongodb.Jep395RecordCodecProvider@25d958c6, com.mongodb.KotlinCodecProvider@4482469c]}, loggerSettings=LoggerSettings{maxDocumentLength=1000}, clusterSettings={hosts=[cowboy5.mongo.REDACTED.com:27017], srvServiceName=mongodb, mode=SINGLE, requiredClusterType=UNKNOWN, requiredReplicaSetName='null', serverSelector='null', clusterListeners='[]', serverSelectionTimeout='30000 ms', localThreshold='15 ms'}, socketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=0, receiveBufferSize=0, proxySettings=ProxySettings{host=null, port=null, username=null, password=null}}, heartbeatSocketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=10000, receiveBufferSize=0, proxySettings=ProxySettings{host=null, port=null, username=null, password=null}}, connectionPoolSettings=ConnectionPoolSettings{maxSize=100, minSize=0, maxWaitTimeMS=120000, maxConnectionLifeTimeMS=0, maxConnectionIdleTimeMS=0, maintenanceInitialDelayMS=0, maintenanceFrequencyMS=60000, connectionPoolListeners=[], maxConnecting=2}, serverSettings=ServerSettings{heartbeatFrequencyMS=10000, minHeartbeatFrequencyMS=500, serverListeners='[]', serverMonitorListeners='[]'}, sslSettings=SslSettings{enabled=true, invalidHostNameAllowed=false, context=javax.net.ssl.SSLContext@40611548}, applicationName='null', compressorList=[], uuidRepresentation=STANDARD, serverApi=null, autoEncryptionSettings=null, dnsClient=null, inetAddressResolver=null, contextProvider=null}
2024-11-08 20:28:52 source > INFO pool-2-thread-1 c.m.i.d.l.SLF4JLogger(info):71 No server chosen by ReadPreferenceServerSelector{readPreference=primary} from cluster description ClusterDescription{type=UNKNOWN, connectionMode=SINGLE, serverDescriptions=[ServerDescription{address=cowboy5.mongo.REDACTED.com:27017, type=UNKNOWN, state=CONNECTING}]}. Waiting for 30000 ms before timing out
2024-11-08 20:28:52 source > INFO cluster-ClusterId{value='672e74845512566e79267c8e', description='null'}-cowboy5.mongo.REDACTED.com:27017 c.m.i.d.l.SLF4JLogger(info):76 Exception in monitor thread while connecting to server cowboy5.mongo.REDACTED.com:27017 com.mongodb.MongoSocketWriteException: Exception sending message
    at com.mongodb.internal.connection.InternalStreamConnection.translateWriteException(InternalStreamConnection.java:714) ~[mongodb-driver-core-4.11.0.jar:?]
    at com.mongodb.internal.connection.InternalStreamConnection.sendMessage(InternalStreamConnection.java:587) ~[mongodb-driver-core-4.11.0.jar:?]
    at com.mongodb.internal.connection.InternalStreamConnection.sendCommandMessage(InternalStreamConnection.java:419) ~[mongodb-driver-core-4.11.0.jar:?]
    at com.mongodb.internal.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:364) ~[mongodb-driver-core-4.11.0.jar:?]
    at com.mongodb.internal.connection.CommandHelper.sendAndReceive(CommandHelper.java:102) ~[mongodb-driver-core-4.11.0.jar:?]
    at com.mongodb.internal.connection.CommandHelper.executeCommand(CommandHelper.java:49) ~[mongodb-driver-core-4.11.0.jar:?]
    at com.mongodb.internal.connection.InternalStreamConnectionInitializer.initializeConnectionDescription(InternalStreamConnectionInitializer.java:130) ~[mongodb-driver-core-4.11.0.jar:?]
    at com.mongodb.internal.connection.InternalStreamConnectionInitializer.startHandshake(InternalStreamConnectionInitializer.java:78) ~[mongodb-driver-core-4.11.0.jar:?]
    at com.mongodb.internal.connection.InternalStreamConnection.open(InternalStreamConnection.java:213) ~[mongodb-driver-core-4.11.0.jar:?]
    at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.lookupServerDescription(DefaultServerMonitor.java:196) ~[mongodb-driver-core-4.11.0.jar:?]
    at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:156) ~[mongodb-driver-core-4.11.0.jar:?]
    at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
Caused by: javax.net.ssl.SSLHandshakeException: Remote host terminated the handshake
    at java.base/sun.security.ssl.SSLSocketImpl.handleEOF(SSLSocketImpl.java:1714) ~[?:?]
    at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1514) ~[?:?]
    at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1421) ~[?:?]
    at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:455) ~[?:?]
    at java.base/sun.security.ssl.SSLSocketImpl.ensureNegotiated(SSLSocketImpl.java:922) ~[?:?]
    at java.base/sun.security.ssl.SSLSocketImpl$AppOutputStream.write(SSLSocketImpl.java:1291) ~[?:?]
    at com.mongodb.internal.connection.SocketStream.write(SocketStream.java:164) ~[mongodb-driver-core-4.11.0.jar:?]
    at com.mongodb.internal.connection.InternalStreamConnection.sendMessage(InternalStreamConnection.java:584) ~[mongodb-driver-core-4.11.0.jar:?]
    ... 10 more
    Suppressed: java.net.SocketException: Broken pipe
        at java.base/sun.nio.ch.SocketDispatcher.write0(Native Method) ~[?:?]
        at java.base/sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:62) ~[?:?]
        at java.base/sun.nio.ch.NioSocketImpl.tryWrite(NioSocketImpl.java:394) ~[?:?]
        at java.base/sun.nio.ch.NioSocketImpl.implWrite(NioSocketImpl.java:410) ~[?:?]
        at java.base/sun.nio.ch.NioSocketImpl.write(NioSocketImpl.java:440) ~[?:?]
        at java.base/sun.nio.ch.NioSocketImpl$2.write(NioSocketImpl.java:819) ~[?:?]
        at java.base/java.net.Socket$SocketOutputStream.write(Socket.java:1195) ~[?:?]
        at java.base/sun.security.ssl.SSLSocketOutputRecord.encodeAlert(SSLSocketOutputRecord.java:81) ~[?:?]
        at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:419) ~[?:?]
        at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:326) ~[?:?]
        at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:467) ~[?:?]
        at java.base/sun.security.ssl.SSLSocketImpl.ensureNegotiated(SSLSocketImpl.java:922) ~[?:?]
        at java.base/sun.security.ssl.SSLSocketImpl$AppOutputStream.write(SSLSocketImpl.java:1291) ~[?:?]
        at com.mongodb.internal.connection.SocketStream.write(SocketStream.java:164) ~[mongodb-driver-core-4.11.0.jar:?]
        at com.mongodb.internal.connection.InternalStreamConnection.sendMessage(InternalStreamConnection.java:584) ~[mongodb-driver-core-4.11.0.jar:?]
        at com.mongodb.internal.connection.InternalStreamConnection.sendCommandMessage(InternalStreamConnection.java:419) ~[mongodb-driver-core-4.11.0.jar:?]
        at com.mongodb.internal.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:364) ~[mongodb-driver-core-4.11.0.jar:?]
        at com.mongodb.internal.connection.CommandHelper.sendAndReceive(CommandHelper.java:102) ~[mongodb-driver-core-4.11.0.jar:?]
        at com.mongodb.internal.connection.CommandHelper.executeCommand(CommandHelper.java:49) ~[mongodb-driver-core-4.11.0.jar:?]
        at com.mongodb.internal.connection.InternalStreamConnectionInitializer.initializeConnectionDescription(InternalStreamConnectionInitializer.java:130) ~[mongodb-driver-core-4.11.0.jar:?]
        at com.mongodb.internal.connection.InternalStreamConnectionInitializer.startHandshake(InternalStreamConnectionInitializer.java:78) ~[mongodb-driver-core-4.11.0.jar:?]
        at com.mongodb.internal.connection.InternalStreamConnection.open(InternalStreamConnection.java:213) ~[mongodb-driver-core-4.11.0.jar:?]
        at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.lookupServerDescription(DefaultServerMonitor.java:196) ~[mongodb-driver-core-4.11.0.jar:?]
        at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:156) ~[mongodb-driver-core-4.11.0.jar:?]
        at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
Caused by: java.io.EOFException: SSL peer shut down incorrectly
    at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:494) ~[?:?]
    at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:483) ~[?:?]
    at java.base/sun.security.ssl.SSLSocketInputRecord.decode(SSLSocketInputRecord.java:160) ~[?:?]
    at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:111) ~[?:?]
    at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1506) ~[?:?]
    at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1421) ~[?:?]
    at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:455) ~[?:?]
    at java.base/sun.security.ssl.SSLSocketImpl.ensureNegotiated(SSLSocketImpl.java:922) ~[?:?]
    at java.base/sun.security.ssl.SSLSocketImpl$AppOutputStream.write(SSLSocketImpl.java:1291) ~[?:?]
    at com.mongodb.internal.connection.SocketStream.write(SocketStream.java:164) ~[mongodb-driver-core-4.11.0.jar:?]
    at com.mongodb.internal.connection.InternalStreamConnection.sendMessage(InternalStreamConnection.java:584) ~[mongodb-driver-core-4.11.0.jar:?]
    ... 10 more

Stack Trace: com.mongodb.MongoSocketWriteException: Exception sending message
    at com.mongodb.internal.connection.InternalStreamConnection.translateWriteException(InternalStreamConnection.java:714)
    at com.mongodb.internal.connection.InternalStreamConnection.sendMessage(InternalStreamConnection.java:587)
    at com.mongodb.internal.connection.InternalStreamConnection.sendCommandMessage(InternalStreamConnection.java:419)
    at com.mongodb.internal.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:364)
    at com.mongodb.internal.connection.CommandHelper.sendAndReceive(CommandHelper.java:102)
    at com.mongodb.internal.connection.CommandHelper.executeCommand(CommandHelper.java:49)
    at com.mongodb.internal.connection.InternalStreamConnectionInitializer.initializeConnectionDescription(InternalStreamConnectionInitializer.java:130)
    at com.mongodb.internal.connection.InternalStreamConnectionInitializer.startHandshake(InternalStreamConnectionInitializer.java:78)
    at com.mongodb.internal.connection.InternalStreamConnection.open(InternalStreamConnection.java:213)
    at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.lookupServerDescription(DefaultServerMonitor.java:196)
    at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:156)
    at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: javax.net.ssl.SSLHandshakeException: Remote host terminated the handshake
    at java.base/sun.security.ssl.SSLSocketImpl.handleEOF(SSLSocketImpl.java:1714)
    at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1514)
    at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1421)
    at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:455)
    at java.base/sun.security.ssl.SSLSocketImpl.ensureNegotiated(SSLSocketImpl.java:922)
    at java.base/sun.security.ssl.SSLSocketImpl$AppOutputStream.write(SSLSocketImpl.java:1291)
    at com.mongodb.internal.connection.SocketStream.write(SocketStream.java:164)
    at com.mongodb.internal.connection.InternalStreamConnection.sendMessage(InternalStreamConnection.java:584)
    ... 10 more
    Suppressed: java.net.SocketException: Broken pipe
        at java.base/sun.nio.ch.SocketDispatcher.write0(Native Method)
        at java.base/sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:62)
        at java.base/sun.nio.ch.NioSocketImpl.tryWrite(NioSocketImpl.java:394)
        at java.base/sun.nio.ch.NioSocketImpl.implWrite(NioSocketImpl.java:410)
        at java.base/sun.nio.ch.NioSocketImpl.write(NioSocketImpl.java:440)
        at java.base/sun.nio.ch.NioSocketImpl$2.write(NioSocketImpl.java:819)
        at java.base/java.net.Socket$SocketOutputStream.write(Socket.java:1195)
        at java.base/sun.security.ssl.SSLSocketOutputRecord.encodeAlert(SSLSocketOutputRecord.java:81)
        at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:419)
        at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:326)
        at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:467)
        ... 14 more
Caused by: java.io.EOFException: SSL peer shut down incorrectly
    at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:494)
    at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:483)
    at java.base/sun.security.ssl.SSLSocketInputRecord.decode(SSLSocketInputRecord.java:160)
    at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:111)
    at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1506)
    ... 16 more

Contribute

marcosmarxm commented 1 week ago

@airbytehq/dbsources can someone take a look into this question?