pmadrigal
commented
8 years ago Hi @moonlightheng ,
I don't know the value of "ListAppDailyStatistic" Do you mean "List(dailyData)"?
Try to create date in other ways. We are inserted dates and timestamps several times and it works.
Here there is and example similar to your code that works for me:
case class AppDailyStatistic(date: java.sql.Date, newUser: Long, launchCount: Long)
val dailyData = AppDailyStatistic(new java.sql.Date(1), 1l, 2l)
val dataFrame: DataFrame = sqlContext.createDataFrame(sc.parallelize(List(dailyData)))
val saveConfig = MongodbConfigBuilder(Map(Host -> List("localhost:27017"), Database -> "test",
Collection -> "app_daily_statistic", SamplingRatio -> 1.0, WriteConcern -> MongodbWriteConcern.Normal,
SplitSize -> 8, SplitKey -> "_id"))
dataFrame.saveToMongodb(saveConfig.build)Please close the issue if your problem is solved or give us more details.
moonlightheng
when I save java.util.Date to mongoDB, it says spark sqlSchema for type java.util.Date is not supported when I change to java.sql.Date it says org.bson.codecs.configuration.CodecConfigurationException: Can't find a codec for class java.sql.Date..
I have tried to ask on stackoverflow but nobody answered, So I have to ask for help here here's my code
case class AppDailyStatistic(date: Date, newUser: Long, launchCount: Long) val dailyData = new AppDailyStatistic(new Date(DateUtils.getStartOfToday.getTime), newUserCount, launchCount) val dataFrame: DataFrame = sqlContext.createDataFrame(sc.parallelize(ListAppDailyStatistic)); val saveConfig = MongodbConfigBuilder(Map(Host -> List("localhost:27017"), Database -> "test", Collection -> "app_daily_statistic", SamplingRatio -> 1.0, WriteConcern -> MongodbWriteConcern.Normal, SplitSize -> 8, SplitKey -> "_id")) dataFrame.saveToMongodb(saveConfig.build)