Config parameters reading from Mongodb

[jamalahmedmaaz]

Hi team,

I am using Java + Spark JobServer + EMR.

have tried both ways. Normal code and sql to read from a collection in mongodb.

Part 1: DataFrame dataFrame = sqlContext.sql("CREATE TEMPORARY TABLE " + tableName + " (emailId STRING, contactId INT, deleted BOOLEAN)" + " USING com.stratio.datasource.mongodb OPTIONS " + " (" + " host x.x.x.x:27017, " + " database xdb, " + " credentials user,database,password;xuser,xdb,xpassword, " + " collection theCollectionBeingRead)");

This throws an

exception.java.lang.RuntimeException: [1.1] failure: ``with'' expected but identifier CREATE found

Part 2:

    DataFrame dataFrame = sqlContext.read()
            .format("com.stratio.datasource.mongodb")
            .option("host", "x.x.x.x:27017")
            .option("database", "xdb").
                    option("credentials", "user,database,password;xuser,xdb,xpassword")
            .option("collection", inputSource.getTableName())
            .load().persist();

This is getting stuck at

[2015-12-27 13:42:59,926] INFO .apache.spark.SparkContext [] [akka://JobServer/user/context-supervisor/mongoContext] - Starting job: aggregate at MongodbSchema.scala:46

I have used spark-redshift connector worked with no issues.

Please guide and point me what i am doing wrong

[pmadrigal]

Hi @jamalahmedmaaz,

Maybe doc is confused, I think your credentials value is incorrect and should be: xuser,xdb,xpassword

The ";" is used when a list of credentials is required to connect.

Besides, first way should have simple quote for the values, like this example:

sqlContext.sql("CREATE TEMPORARY TABLE mongoTable USING com.stratio.datasource.mongodb OPTIONS (host 'host:port', database 'highschool', collection 'students', credentials 'xuser,xdb,xpassword')");

Let us know if your problem is solved.

Thanks!

[jamalahmedmaaz]

Hi @pmadrigal

Tried both it is not working it is stuck at

INFO .apache.spark.SparkContext [] [akka://JobServer/user/context-supervisor/mongoContext] - Starting job: aggregate at MongodbSchema.scala:46

please guide in the right direction.

[anand-singh]

Hi @pmadrigal ,

ERROR : Caused by: com.mongodb.MongoException: not authorized for query on spark-mongo-1.sample_1

Map(
      Host -> List(s"${host}:${port}"),
      Database -> database,
      Credentials -> s"${user},${database},${password}",
      Collection -> collection,
      SamplingRatio -> 1.0,
      WriteConcern -> MongodbWriteConcern.Normal) ++ additional
  }

please guide in the right direction.

[pmadrigal]

Answered on #68