meltano / hub

The single source of truth for all Meltano plugins, including all available Singer Taps and Targets: https://hub.meltano.com
https://hub.meltano.com
50 stars 68 forks source link

Should Hub have a relational database backend #208

Open MeltyBot opened 2 years ago

MeltyBot commented 2 years ago

Migrated from GitLab: https://gitlab.com/meltano/hub/-/issues/212

Originally created by @aaronsteers on 2022-03-10 22:53:48


This keeps coming up so I want to address it. Sounds we're mostly expecting we'll eventually add an RDMS but for me, this has always been an architectural antipattern. (Admittedly I'm not a front end dev.)

At Amazon, traditional RDMS's were expressly forbidden for any production workloads that needed to scale, so I'm admittedly pre-biased against RDBM's and OLTP use cases for the Hub.

Would be helpful to understand what requirements we feel like would be benefited from an RDMS.

If we do have use cases that will require RDMS backend, I'd like to understand those better so I can update my own expectations and so we can start designing the long-term solution that we need.

MeltyBot commented 2 years ago

View 2 previous comments from the original issue on GitLab