search

datastacktv / data-engineer-roadmap

Roadmap to becoming a data engineer in 2021
https://datastack.tv
12.2k stars 1.31k forks source link

Pub/Sub messaging #8

Closed mlliarm closed 3 years ago

mlliarm commented 3 years ago

Hello,

Not sure how to edit a png to add a new pull request, so I'm creating this issue.

I believe GCP's Pub/Sub messaging system deserves to be under the "Messaging" section too.

alexandraabbas commented 3 years ago

Hey @mlliarm, thanks for the suggestion! In my opinion if we include Google's PubSub then we should include Amazon SQS and/or SNS as well as Azure Service Bus. But that would be overwhelming so that's why I only included the open source options. Why do you think it would be necessary to include cloud based options?

Based on Google trends Azure Service Bus is the most popular.

mlliarm commented 3 years ago

@alexandraabbas Hey, thanks for the reply !

Regarding your question, I believe it'd be important to add the relevant cloud options as it seems to me that this is the age where many people move their on premise systems on the cloud. There are some advantages of having your pipelines on the cloud, I believe, where part of your system or whole of it is managed by the cloud provider (see difference between IaaS, PaaS and SaaS).

I understand that the infographic might get more overwhelming if all the cloud solutions would be added, but on the other hand I think that the prospect data engineer should know all the available options where he can choose from.