Citus: Performance test REST API

hashgraph / hedera-mirror-node

Hedera Mirror Node archives data from consensus nodes and serves it via an API

Apache License 2.0

128 stars 108 forks source link

Citus: Performance test REST API #4655

Closed steven-sheehy closed 1 month ago

steven-sheehy commented 1 year ago

Problem

We need to test the performance of CitusDB with 100B transactions when using the REST API

Solution

Depends on #5750
Run the k6 performance test
Compare with Cloud SQL mainnet staging against same version and similar timeframe
Load transactions into CitusDB maxing out TPS using RecordFileParserPerformanceTest
Verify gRPC API at 10K TPS
Repeat at 25B, 50B, 75B, and 100B

Alternatives

No response

mgoelswirlds commented 1 year ago

I think we should list out other tasks that need to be done as part of this ticket and create separate tasks to be able to divide the work properly.

MarcKriguerAtHedera commented 1 year ago

I found this article on how to configure a multi-node Citus environment -- it seems to suggest we use an Ubuntu GKE setup. But until I can get the RecordFileParserPerformanceTest to work locally, it seems a little premature to start getting the new environments ready to use.

MarcKriguerAtHedera commented 1 year ago

Added Notion page https://www.notion.so/swirldslabs/Evaluating-Citus-DB-Performance-938068a629b84fe1920bd81f24310ed0 specifying all changes made in evaluating single-node Citus environment, in preparation to make (most of) the same changes (on a pre-prod VM) to run against the multi-node Citus environment.

steven-sheehy commented 1 month ago

This has been completed and done in an ongoing basis