m5 instances for ScyllaDB are not the recommended optimized instance family type

[roydahan]

In all of your ScyllaDB benchmarks, you've used m5 instance type family. (e.g https://github.com/benchANT/database-ranking/blob/main/CRUD-general-purpose/2022/ranking_batch-0_scylladb-451-vanilla-os-large-large-awsec2-2022_0_1NkNG04T/dbms_data_hardware_facts.json)

ScyllaDB recommends to use "Storage Optimized" instance types (as you used in your blog Mongo vs Cassandra). i3, i3en or the new i4i instances are much more suitable option to benchmark ScyllaDB with.

[seybi87]

Hi @roydahan thanks for the feedback and we absolutely agree on that the used general purpose instances across all cloud providers do not reflect a production deployment for ScyllaDB (nor for any other DBMS where performance matters).

The currently available benchmarks in the ranking are only intended to demonstrate a very basic performance baseline and do not reflect a production setup as also stated in the disclaimer.

Of course, benchANT support all the high performance instance types and storage types across the supported cloud providers and we are happy to include them in our ranking. If you are interested in more a production like ranking with the storage optimized instance types, just let us know.

[roydahan]

Hi @seybi87, where can I find the benchmark sources for this blog https://benchant.com/blog/mongodb-vs-cassandra? In that one you used i3.xlarge and I thought it would be nice to repeat it with Scylla and compare results.

[seybi87]

Hi @roydahan this data was not yet published but I have uploaded it right now into this repository: https://github.com/benchANT/mongodb-vs-cassandra-blog

Sure, repeating the benchmarks with ScyllaDB would be interesting, we will see how this fits into our schedule.

In case you have a pressing interest in these benchmarks, just let us know.

[roydahan]

Thanks @seybi87 ! We're going to repeat it ourselves in order to test be able to compare ourselves to the results you published.