Reduce overall complexity of the DT

[misterious-mister-x]

There are several ways to achieve this. One is to move more things into Drill Downs and reduce amount of services directly exposed. Please take a look at the picture below.

Thoughts?

[misterious-mister-x]

New version attached. Please comment.

[elsantorini]

[misterious-mister-x]

@elsantorini thanks for thoughtful review! My answers below.

• Why 24 hours - agree (probably from the past experience) - will change it
• Event Triggers - not sure what you mean here - I am referring to a situation when we are doing combination of upsert and append (on top) and combination of sequential reads via Change Feed and Lookups (when both are needed).
• I disagree with rephrasing (please refer to the statement above). This is something which helps us to compare EH with Cosmos on the certain workloads. Cosmos DB is described on the right hand side.
• Kafka protocol support is one of the main differentiators for EH especially comparing to Service Bus and IoT Hub so tend to disagree.
• Agree with the comment on Web Apps - indeed seems that I have missed this when was compiling current version.
• TSI / ADX & PBI / AAS - fully agree - completely forgot about it.
• On Graph - it is a good question. We need to make some extra comparison / will ask some guys for their opinion.

[elsantorini]

• Event Triggers - I mean that another read pattern might be a case where you read the changes made in the DB and you trigger other actions.

• ok, no prob with the rephrasing, just wanted to simplify because basically it uses phrases that are mentioned in the two previous steps (append combined with upsert & storing for long term)

• For EH vs IoT Hub vs Service Bus- there are other differentiators like max size of throughput supported, or size of message etc. I find these more relevant to write/read patterns than support of a protocol.

[misterious-mister-x]

"For EH vs IoT Hub vs Service Bus- there are other differentiators like max size of throughput supported, or size of message etc. I find these more relevant to write/read patterns than support of a protocol."

The thing is that for the overlap below 256 KB of message size and ~8 MBps in (which is where most of the workloads will find themselves) there will be no other differentiator. That said just size of the message and max throughput cannot be identified as safe criteria. Also when it comes to difference between Stream and Queue consistency and functionality of two are much more important (as you typically split streams and queues and do not store / process everything via one gigantic instance).

Not arguing - just explaining the logic here. Let's also discuss this separately.

[misterious-mister-x]

Event Triggering is more on the processing side, no? We need to review this. In case we add Event Triggering it is fair to add Event Grid explicitly with all the integrations.

[misterious-mister-x]

• Why 24 hours - removed
• I have added Event Grid and new read pattern (tentatively)
• Rephrasing for Cosmos DB done: "Data can be updated and later should be stored and queried (not sequentially read) for a period longer than EH retention"
• I have rephrased that so EH is "Streaming Semantics" while Service Bus "Queue semantics or Hybrid / FIFO / Exact Once"

[misterious-mister-x]

WebApps fixed

[misterious-mister-x]

TSI / ADX and PBI / AAS fixed

[misterious-mister-x]

Graph changed (not sure if fixed ;)

[misterious-mister-x]

[elsantorini]

Great :) Love it! Two points only:

1. In the event triggering square, you mention ~100s RPS and on the following two squares you mention 5,000 events per sec in the Low/Medium side. Requests in that case mean small batches?
2. Some difference between Azure SQL and Gremlin can be on the write pattern the following:

• on Azure SQL you can't store multi-parents for a node
• on Azure SQL many new relationships updates are not supported

[misterious-mister-x]

Thanks for all the feedback. Based on discussions & feedback I have crafted the newer version.