search

microsoft / fabricnotes

Simple drawings illustrating the main concepts of Microsoft Fabric to empower anyone to build stuff on Fabric.
https://microsoft.github.io/fabricnotes
Creative Commons Attribution 4.0 International
88 stars 17 forks source link

[Note] Event-driven consumption & writing to delta tables #13

Open robcube opened 1 year ago

robcube commented 1 year ago

What concept do you need more clarity on? Event-driven architecture using Event Hub as the source

Any reference explaining that concept Event Hub integration: using Spark to ingest Event Hub payloads into Delta tables, using Spark to create Event Hub messages

Have you checked the list of issues to see if there is a similar suggestion?

cmaneu commented 1 year ago

Hi @robcube, Thanks for your suggestion! Do you absolutely want to use Spark for the ingestion? With Synapse Realtime within Fabric, there are simpler options (assuming you don't need specifc transformations)

Marcus-Brubaker-Prot commented 1 year ago

Hi @robcube, Thanks for your suggestion! Do you absolutely want to use Spark for the ingestion? With Synapse Realtime within Fabric, there are simpler options (assuming you don't need specifc transformations)

Include the different options. Perhaps something like "Event driven architecture combining Event hub and low-code options <Synapse Realtime, X, Y, Z>" and "Able to use Spark for <custom | complex transformations, other use cases> by leveraging <Spark feature | function | plug-in | etc.>"