Stratio / Spark-MongoDB

Spark library for easy MongoDB access
http://www.stratio.com
Apache License 2.0
307 stars 99 forks source link

user & pwd #42

Closed nirby82 closed 8 years ago

nirby82 commented 8 years ago

Hi,

when I connect to mongodb from terminal I pass User, Password and AuthenticationDatabase

how can i pass these arguments through this connector?

thanks

pmadrigal commented 8 years ago

Hi!

You have to set the required parameters:

Moreover, in your special case you have to set credential parameter.

So, your builder should be as follows:

val builder = MongodbConfigBuilder(Map(Host -> List("localhost:27017"), Database -> "databaseName", Collection -> "collectionName", Credentials -> "user,database,password"))
val readConfig = builder.build()

if you have a big collection and you know the collection schema, it is better to use it, else, add a small SamplingRatio to the builder.

For small collections this doesn't matter.

You can see complete examples in our documentation.

nirby82 commented 8 years ago

thanks

seems like a connection was made because i am no longer getting a connection error. now I am receiving this error: Exception in thread "main" java.lang.ClassCastException: java.lang.String cannot be cast to scala.collection.TraversableLike at com.stratio.provider.mongodb.partitioner.MongodbPartitioner.(MongodbPartitioner.scala:42) at com.stratio.provider.mongodb.MongodbRelation.(MongodbRelation.scala:51) at com.stratio.provider.mongodb.MongodbContext.fromMongoDB(mongodbFunctions.scala:43) at mongodb$.main(mongodb.scala:43) at mongodb.main(mongodb.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)

pmadrigal commented 8 years ago

Which version of our project are you using?

nirby82 commented 8 years ago

0.9.0

pmadrigal commented 8 years ago

Sorry but i can't reproduce your error, credentials are working for me.

Let me know more about your code.

Are you using master branch, right?

This is a simple example with java, and works for me:

public class Mongo {

    public static void main(String[] args) {

        JavaSparkContext sc = new JavaSparkContext("local[2]", "test mongo java"); 
        SQLContext sqlContext = new org.apache.spark.sql.SQLContext(sc);
        Map options = new HashMap();
        options.put("host", "localhost:27017");
        options.put("database", "highschoolCredentials");
        options.put("collection", "students");
        options.put("credentials", "user,highschoolCredentials,password");

        DataFrame df = sqlContext.read().format("com.stratio.provider.mongodb").options(options).load();
        df.show();

    }
}