microsoft / sql-spark-connector

Apache Spark Connector for SQL Server and Azure SQL
Apache License 2.0
273 stars 116 forks source link

Py4JJavaError: An error occurred while calling o866.load. #232

Open vladimirteddy opened 1 year ago

vladimirteddy commented 1 year ago

My code:

my_spark = SparkSession \
    .builder \
    .appName("myApp") \
    .config("spark.mongodb.read.connection.uri", "mongodb+srv://admin:123456789@carz-etl.podftog.mongodb.net/mrt_carz.mrt_carz_bookings?retryWrites=true&w=majority") \
    .config("spark.mongodb.read.connection.uri", "mongodb+srv://admin:123456789@carz-etl.podftog.mongodb.net/mrt_carz.mrt_carz_bookings?retryWrites=true&w=majority") \
    .getOrCreate()

df = my_spark.read.format("com.mongodb.spark.sql.DefaultSource")\
.option("uri", "mongodb+srv://admin:123456789@carz-etl.podftog.mongodb.net/mrt_carz.mrt_carz_bookings?retryWrites=true&w=majority")\
.option("database", "mrt_carz")\
.option("collection", "mrt_carz_bookings")\
.load()
df.select('*').where(col("name") == "mrt_carz_bookings").show()

And result: Py4JJavaError: An error occurred while calling o866.load. : com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting for a server that matches com.mongodb.client.internal.MongoClientDelegate$1@fd730d0. Client view of cluster state is {type=REPLICA_SET, servers=[{address=xxx.podftog.mongodb.net:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketWriteException: Exception sending message}, caused by {javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target}, caused by {sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target}, caused by {sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target}}, {address=ac-u6vtlhr-shard-00-02.podftog.mongodb.net:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketWriteException: Exception sending message}, caused by {javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target}, caused by {sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target}, caused by {sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target}}, {address=ac-u6vtlhr-shard-00-00.podftog.mongodb.net:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketWriteException: Exception sending message}, caused by {javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target}, caused by {sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target}, caused by {sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target}}] at com.mongodb.internal.connection.BaseCluster.createTimeoutException(BaseCluster.java:408) at com.mongodb.internal.connection.BaseCluster.selectServer(BaseCluster.java:123) at com.mongodb.internal.connection.AbstractMultiServerCluster.selectServer(AbstractMultiServerCluster.java:54) at com.mongodb.client.internal.MongoClientDelegate.getConnectedClusterDescription(MongoClientDelegate.java:152) at com.mongodb.client.internal.MongoClientDelegate.createClientSession(MongoClientDelegate.java:102) at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.getClientSession(MongoClientDelegate.java:282) at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.execute(MongoClientDelegate.java:186) at com.mongodb.client.internal.MongoDatabaseImpl.executeCommand(MongoDatabaseImpl.java:194) at com.mongodb.client.internal.MongoDatabaseImpl.runCommand(MongoDatabaseImpl.java:163) at com.mongodb.client.internal.MongoDatabaseImpl.runCommand(MongoDatabaseImpl.java:158) at com.mongodb.spark.MongoConnector.$anonfun$hasSampleAggregateOperator$1(MongoConnector.scala:237) at com.mongodb.spark.MongoConnector.$anonfun$withDatabaseDo$1(MongoConnector.scala:174) at com.mongodb.spark.MongoConnector.withMongoClientDo(MongoConnector.scala:157) at com.mongodb.spark.MongoConnector.withDatabaseDo(MongoConnector.scala:174) at com.mongodb.spark.MongoConnector.hasSampleAggregateOperator(MongoConnector.scala:237) at com.mongodb.spark.rdd.MongoRDD.hasSampleAggregateOperator$lzycompute(MongoRDD.scala:221) at com.mongodb.spark.rdd.MongoRDD.hasSampleAggregateOperator(MongoRDD.scala:221) at com.mongodb.spark.sql.MongoInferSchema$.apply(MongoInferSchema.scala:68) at com.mongodb.spark.sql.DefaultSource.constructRelation(DefaultSource.scala:97) at com.mongodb.spark.sql.DefaultSource.createRelation(DefaultSource.scala:50) at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:350) at org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:274) at org.apache.spark.sql.DataFrameReader.$anonfun$load$3(DataFrameReader.scala:245) at scala.Option.getOrElse(Option.scala:189) at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:245) at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:174) at sun.reflect.GeneratedMethodAccessor75.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244) at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) at py4j.Gateway.invoke(Gateway.java:282) at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132) at py4j.commands.CallCommand.execute(CallCommand.java:79) at py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182) at py4j.ClientServerConnection.run(ClientServerConnection.java:106) at java.lang.Thread.run(Thread.java:745)

Please help me any solution?

dazfuller commented 1 year ago

That's the MongoDB connector, this repo is for the Microsoft SQL Server connector.

Looking at the error I'd say there's an SSL setup issue as it can't find a valid SSL cert for the server you're trying to access