search

Stratio / Spark-MongoDB

Spark library for easy MongoDB access
http://www.stratio.com
Apache License 2.0
307 stars 96 forks source link

splitkey not working #171

Open bbnsumanth opened 7 years ago

bbnsumanth commented 7 years ago

I'm using the following code to get data using a split key on pid. But unfortunately, spark is fetching all the data from MongoDB collection.Can someone say me where I'm going wrong. val mongoConfig = MongodbConfigBuilder( Map( Host -> mongoHost, Database -> mongoDatabase, Collection -> mongoCollection, SamplingRatio -> 1.0, WriteConcern -> "normal", SplitSize -> "5", SplitKey -> "pid", SplitKeyMin -> "357", SplitKeyMax -> "368", SplitKeyType -> "int" ) ).build() val df = spark.sqlContext.fromMongoDB(mongoConfig)

bbnsumanth commented 7 years ago

I found the reason for this problem,there is no index on "pid" field in my mongoCollection.I would be very helpful if this is included in the documentation.