Sequoia Capital: Why We Moved from Elasticsearch to Rockset

[Sunt-ing]

https://rockset.com/blog/sequoia-capital-elasticsearch-to-rockset/

[Sunt-ing]

Usage: internal analytical applications in Sequoia Capital.

Replacement Reason:

• Rockset supports SQL which makes engineers much more productive as well as less error-prone than using Elasticsearch DSL.
• Rockset indexes data faster, bringing higher data recency. A 4-5 hours data indexing job with Elasticsearch can be finished in 15-30 minutes with Rockset.
• Elasticsearch doesn’t support joins and thus requires denormalizing data.
  • It can take a week to set up a Spark job to denormalize each data set.
  • Denormalization brings as about 100x space amplification: Data that would occupy 1 TB in Elasticsearch now takes up 10 GB in Rockset.
• Rockset requires much less parameter tuning than using Elasticsearch and has much less downtime.

Other considered alternatives:

• Postgres is selective about the data users put into it, potentially limiting the datasets users bring into their apps.
• Snowflake and Amazon Athena are way slower for powering apps.
• All the NoSQL alternatives required learning something different from SQL.