search

topcoder-platform / thrive-submit

Thrive article tracking
6 stars 5 forks source link

[$100] Part 2: Databricks Series - Introduction to Streaming : Spark & DataBricks #344

Closed bhagvanarch closed 3 years ago

bhagvanarch commented 3 years ago

This article will be about Streaming

I will add the outline soon.

I am planning to write a series of articles related to DataBricks.

  1. Introduction to Machine Learning: using DataBricks
  2. Introduction to Streaming: Spark & DataBricks
  3. Job Processing with DataBricks
  4. Creating a Spark Cluster in DataBricks Workspace
  5. Data Transformations with DataBricks
  6. Azure DataBricks ETL
  7. Working with Azure DataBricks NoteBook
  8. Data Security: Azure DataBricks
kkartunov commented 3 years ago

@bhagvanarch great. Please, tag your working articles with [WIP] as per the repo rules https://github.com/topcoder-platform/thrive-submit#step-1-verify-your-article-is-unique-and-not-wip

bhagvanarch commented 3 years ago

The outline of the article is as below:

  1. DataBricks Streaming Architecture
  2. DataBricks Data Lake
  3. Data Frames & Data Sets
  4. Databricks Account Creation
  5. Cluster Creation
  6. Input Data Creation
  7. Data Stream Processing
  8. Data Aggregation
  9. Data WareHouse
bhagvanarch commented 3 years ago

The draft article is created for your review.

https://docs.google.com/document/d/13C-b7HNrt5sb9_fYr6465nfDOlxUft4J8NhvCz2wf44/edit?usp=sharing

harshitmehta15 commented 3 years ago

@kkartunov good to publish :)

kkartunov commented 3 years ago

Challenge https://www.topcoder.com/challenges/8d5ad635-fdc1-447a-84d3-3cfd93497f43 has been created for this ticket.

This is an automated message for kiril.kartunov via Topcoder X

kkartunov commented 3 years ago

Challenge https://www.topcoder.com/challenges/8d5ad635-fdc1-447a-84d3-3cfd93497f43 has been assigned to bhagvanarch.

This is an automated message for kiril.kartunov via Topcoder X

kkartunov commented 3 years ago

Published https://www.topcoder.com/thrive/articles/introduction-to-streaming-spark-and-databricks Thanks, @bhagvanarch

kkartunov commented 3 years ago

Payment task has been updated: https://www.topcoder.com/challenges/8d5ad635-fdc1-447a-84d3-3cfd93497f43 Payments Complete Winner: bhagvanarch Copilot: kiril.kartunov Challenge 8d5ad635-fdc1-447a-84d3-3cfd93497f43 has been paid and closed.

This is an automated message for kiril.kartunov via Topcoder X