document choice of bigquery

subugoe / leine

Data Pipelines for @subugoe/wag

https://subugoe.github.io/leine

MIT License

1 stars 0 forks source link

Open maxheld83 opened 3 years ago

maxheld83 commented 3 years ago

I think we can describe this choice in a bit more detail:

we (far) outstrip a single node, but we don't want to manage Cassandra/Hadoop/Spark.
we need columnar storage / OLAP, but we'd like familiar SQL (again, that kicks out of Cassandra)

maxheld83 commented 3 years ago

need to comment on / compare to aws redshift

maxheld83 commented 3 years ago

I think these are the relevant competitors:

maxheld83 commented 3 years ago

this article is a good overview, suggests that BigQuery is the "most serverless": https://www.altexsoft.com/blog/snowflake-redshift-bigquery-data-warehouse-tools/

I'm guessing this relates to having to manage nodes and sharding (not necessary in BigQuery).