search

dfe-analytical-services / analysts-guide

A static website to hold guidance, helpful links and code examples for analysts working in DfE.
https://dfe-analytical-services.github.io/analysts-guide/
MIT License
5 stars 3 forks source link

Add diagram to show delta lake - cluster - IDE relationship in Databricks #72

Open jen-machin opened 3 months ago

jen-machin commented 3 months ago

Is your feature request related to a problem? Please describe. Working out what each thing is and does can be difficult so a diagram might make it clearer for people new to the platform

Delta lake: where the data lives and is structured Cluster: what you use to connect your IDE to the data Databricks: IDE / platform that allows you to interrogate or work with the data

Additional context image

cjrace commented 3 months ago

Worth making a comparator to SQL server before using SSMS and also R Studio

jen-machin commented 3 months ago

Analysts Guide: add diagram to show delta lake > cluster > Databricks

jen-machin commented 3 months ago

Think there's also something different here depending on how you're planning on using the data from the Delta Lake

So if using the Databricks platform/SQL editor in the browser it's Delta Lake ⬅️ Cluster ⬅️ Databricks analysis platform

If using RStudio it's Delta Lake ⬅️ Cluster ⬅️ ODBC connection ⬅️ RStudio