Closed utterances-bot closed 3 weeks ago
Brother vey deep and informative article, I need on help.I also want to go deep in Spark n Pyspark, but only way to do i see is if i get a project in my job to work upon it? would you be able to recommend me some other way to get expertise in Spark ?.not just going through some courses. Any help would be highly appreciated
Hi @gogi2811, thanks for the feedback! As for your question, I am thinking of several options:
Hi @luminousmen thank you very much for your help and guidance. I'll start with Twitter analysis project and take it deep. I'll keep coming back to you for guidance. Thanks a lot
Well summarized!
Spark Tips. Partition Tuning - Blog | luminousmen
Data partitioning is critical to data processing performance especially for large volumes of data processing in Spark. Here are some partitioning tips
https://luminousmen.com/post/spark-tips-partition-tuning