nurzhannogerbek
commented
5 years ago Well, finally I found the correct way to solve my task. Here below is working code:
import spark.implicits._
val df1 = Seq(
("2019-01-01 00:00:00", "7056589658"),
("2019-02-02 00:00:00", "7778965896")
).toDF("DATE_TIME", "PHONE_NUMBER")
df1.show()
val df2 = Seq(
("2019-01-01 01:00:00", "194.67.45.126"),
("2019-02-02 00:00:00", "102.85.62.100"),
("2019-03-03 03:00:00", "102.85.62.100")
).toDF("DATE_TIME", "IP")
df2.show()
df1.coalesce(1).write
.format("com.crealytics.spark.excel")
.option("dataAddress", "'My Sheet1'!A1:Z1000000")
.option("useHeader", "true")
.option("dateFormat", "yy-mmm-d")
.option("timestampFormat", "mm-dd-yyyy hh:mm:ss")
.mode("append")
.save("/hdd/home/NNogerbek/data.xlsx")
df2.coalesce(1).write
.format("com.crealytics.spark.excel")
.option("dataAddress", "'My Sheet2'!A1:Z1000000")
.option("useHeader", "true")
.option("dateFormat", "yy-mmm-d")
.option("timestampFormat", "mm-dd-yyyy hh:mm:ss")
.mode("append")
.save("/hdd/home/NNogerbek/data.xlsx")QUESTION: Is it necessary to set cells in dataAddress option? Let's say I want to set only name of the sheet?! Is it possible? I tried next options: .option("dataAddress", "'My Sheet1'") and .option("dataAddress", "My Sheet1") but both of them raise error.
nightscape
jain95Garima
In
Scala/Sparkapplication I created two different DataFrame. Spark:2.3.3and Scala:2.11.8.My task is to create one excel file with two sheet for each DataFrame. I decided to usespark-excellibrary (0.12.0) but I am little bit confused.Here is my code:
In
MesosI notice that my pretty simple code created more than 200 jobs. A new job is created every minute. All of them run the code of DataLocator.scala file which is inside thespark-excellibrary. In my opinion, this is a very strange behavior.As far as I understand the future excel file is saved in the hdfs file system, right? I need to set the path of the future excel file in
.save()method, right? Also I don't understand what format should be indataAddressoption? Is there any error in my code?