search

hashgraph / hedera-mirror-node

Hedera Mirror Node archives data from consensus nodes and serves it via an API
Apache License 2.0
131 stars 109 forks source link

Investigate and optimize auto vacuum settings #7422

Closed xin-hedera closed 8 months ago

xin-hedera commented 10 months ago

Description

Importer record file parsing duration spikes when ingesting extremely large record files with > 50k HCS topic messages in perfnet, temporarily disabling autovacuum on topic_message table can reduce the time to less than 2 seconds, however with certain cloudsql instances it doesn't help at all.

On the other side, the current autovacuum settings for most insert only tables are tuned for balanced writes and reads, we need to investigate the impact to database reads if we need to change the settings to favor writes over reads.

Steps to reproduce

  1. Restore the database from a recent backup before the TPS spike
  2. Run importer and observe parsing duration when it starts to ingest large record files

Additional context

No response

Hedera network

other

Version

v0.95.0-SNAPSHOT

Operating system

None

steven-sheehy commented 8 months ago

We now believe this is due to a CPU config change that did not grant the importer enough CPU to handle the transaction spike in testnet. This has since been re-configured correctly.