bartosz25 / spark-scala-playground

Sample processing code using Spark 2.1+ and Scala
50 stars 25 forks source link

Do you have plan to write an article about cube and rollup #3

Closed bithw1 closed 5 years ago

bithw1 commented 5 years ago

Hi @bartosz25

I am reading your posts http://www.waitingforcode.com/apache-spark-sql

I would ask whether you have plan to write something about cube,rollup and grouping_id, I find it is kind of hard to understand, it would be very helpful if you investigate and write something, :-) Thank you very much

bithw1 commented 5 years ago

After spending some time, I think I have got a basic understanding about cube and rollup, so i would close this issue.

bartosz25 commented 5 years ago

Hi @bithw1,

Thanks for your question :+1: . I've never had to use cube and rollup but after a quick research they seem to be interesting concepts. I'll try to explore them during the week 08-14/10 and publish one or 2 weeks after.

If you have some other demands, feel free to share :)

Best regards, Bartosz.

bithw1 commented 5 years ago

Thanks @bartosz25 very much for the reply and the effort to write the articles.

btw, cube is the core concept for the popular open source project Kylin(http://kylin.apache.org/), which is used for multi-dimensional analysis (OLAP),

And I would let you know if I have new demands by opening new issues here., and thanks again, I really learned a lot by reading and running the test cases in your articles.

bartosz25 commented 5 years ago

Hi @bithw1

Thanks again for the idea ! It was interesting to discover grouping sets. I've just published a post about them: http://www.waitingforcode.com/apache-spark-sql/grouping-sets-apache-spark-sql/read

Best regards, Bartosz.

bithw1 commented 5 years ago

Thanks @bartosz25, I really like your posts, and learned a lot from you, thank you very much @bartosz25

bithw1 commented 5 years ago

Hi @bartosz25 Would you please post the code to github? I didn't find it there,thank you.

bartosz25 commented 5 years ago

@bithw1 I've just pushed it here: https://github.com/bartosz25/spark-scala-playground/blob/a4fcc1cac500abe98275ab4e4565b522c5c1be5b/src/test/scala/com/waitingforcode/sql/GroupingSetsTest.scala

bithw1 commented 5 years ago

Sure, thank you @bartosz25!