Problem: a healthy Indexer takes unnecessarily long (20 seconds) to start

[robinbb]

When an Indexer which has no otherwise-known problem starts with an already constructed database of about 58G, it takes 20 seconds to start. The logs reveal what it is doing:

2024-07-10T16:23:24.065594+00:00 - DEBUG Building mina indexer configuration
2024-07-10T16:23:24.065834+00:00 - DEBUG Ensuring blocks directory exists in "/mnt/mina-indexer-prod/blocks-340000"
2024-07-10T16:23:24.065854+00:00 - DEBUG Ensuring staking ledgers directories in "/mnt/mina-indexer-prod/staking-ledgers"
2024-07-10T16:23:24.068366+00:00 - INFO Using default mainnet genesis ledger
2024-07-10T16:23:24.071368+00:00 - DEBUG Creating a new mina indexer database in "/mnt/mina-indexer-prod/db/0.7.1-340000"
2024-07-10T16:23:30.125596+00:00 - DEBUG INDEXER_VERSION file exists. Checking for compatability
2024-07-10T16:23:30.125673+00:00 - INFO Starting the web server listening on 0.0.0.0:8080
2024-07-10T16:23:30.125685+00:00 - INFO Shutting down primary database instance
2024-07-10T16:23:30.125741+00:00 - INFO Initializing new mina indexer
2024-07-10T16:23:30.127246+00:00 - INFO Syncing indexer state from db at "/mnt/mina-indexer-prod/db/0.7.1-340000"
2024-07-10T16:23:30.127343+00:00 - INFO Parsing staking ledgers in /mnt/mina-indexer-prod/staking-ledgers
2024-07-10T16:23:30.127778+00:00 - DEBUG Looking for witness tree root block
2024-07-10T16:23:37.805474+00:00 - DEBUG Witness tree root block (length 339990): 3NLR8h4sNNbp2zwC6YhUwd75LsaS4z64sJM3Dv8wgY6NCvVWaaYD
2024-07-10T16:23:44.271234+00:00 - INFO Adding new blocks from "/mnt/mina-indexer-prod/blocks-340000"
2024-07-10T16:23:44.271306+00:00 - INFO Reporting every 180s or 1000 blocks
2024-07-10T16:23:44.295862+00:00 - INFO Adding block to witness tree mainnet (length 339991): 3NLcYCrjHxwauQN8MB8x32MFk28wAkpotBU9h9SpDEig7EQo98Md
2024-07-10T16:23:44.312414+00:00 - INFO Adding block to witness tree mainnet (length 339992): 3NKPVXpww8gKjWEbDyTuCW9PM4LkgjzpBQk7hkH13rzuYPGWHWu2
2024-07-10T16:23:44.329674+00:00 - INFO Adding block to witness tree mainnet (length 339992): 3NKj4KBor9cc1z8Su9tXyge744HvMgPH9UfumRZKPiVdz2gtMa1D
2024-07-10T16:23:44.346476+00:00 - INFO Adding block to witness tree mainnet (length 339992): 3NKtZuyX3axRwaD9PXu3M3caeXeSVALSvJKKRAtncNJ974JaRqjV
2024-07-10T16:23:44.363808+00:00 - INFO Adding block to witness tree mainnet (length 339992): 3NLEvskGqDye38yCNDQZMbx252KUtFWq416VdwV9aEmpUNTjAjpY
2024-07-10T16:23:44.368356+00:00 - INFO Adding block to witness tree mainnet (length 339993): 3NKeEMjpdSHbep3GiatqQgTJQSMKDGM2u9YFEqnJumHPuD1BJGZV
2024-07-10T16:23:44.371617+00:00 - INFO Adding block to witness tree mainnet (length 339993): 3NL8y3QAHiAgfCqAB6FeUjf8KJopkVPAhGKKF18Jvogk97utbH4j
2024-07-10T16:23:44.377931+00:00 - INFO Adding block to witness tree mainnet (length 339993): 3NL9bEk4r58zV2umQMZ7rDEmKXwuy8HoKv2c6FtvNK2y79ZrQsCB
2024-07-10T16:23:44.400376+00:00 - INFO Adding block to witness tree mainnet (length 339994): 3NKJNJ67473XjQtHvjTxa3v3YWVjdMvvU3FDy8F6ENsHJ8RjxDjD
2024-07-10T16:23:44.406672+00:00 - INFO Adding block to witness tree mainnet (length 339994): 3NLPemFRP9kHA3HSBQ28weSdbxhbCECzDcKiSYBF6FRn5KdXA1f1
2024-07-10T16:23:44.419639+00:00 - INFO Adding block to witness tree mainnet (length 339995): 3NKLMzf4f9UU9npgA8xYXn7zaHhVDU6m3hyTKmnak3YexPLLLuBL
2024-07-10T16:23:44.429939+00:00 - INFO Adding block to witness tree mainnet (length 339995): 3NKt5bvvv9wRYpFLkUQEaCvPprhndoYA1gjVkwKFiDeHg9ZcG6P3
2024-07-10T16:23:44.438520+00:00 - INFO Adding block to witness tree mainnet (length 339995): 3NLkVY42R7h4pmoU1FSHytRAAWRDwNRc4V7as1ULFTrtPrpYv8Gd
2024-07-10T16:23:44.456077+00:00 - INFO Adding block to witness tree mainnet (length 339996): 3NKPKwsvnbUs2hvSBBUggG3FMjx38SgPhjpQzoMVW2K2YpjPsYxy
2024-07-10T16:23:44.472553+00:00 - INFO Adding block to witness tree mainnet (length 339996): 3NKXAwBLtwRrCcnsKKUYCRtivj7YWSVvBz6Kh7cWAnosPZpKX8hs
2024-07-10T16:23:44.489170+00:00 - INFO Adding block to witness tree mainnet (length 339996): 3NKfM14LRdQup1g9oUew9k2KMXfpSiPZkJnCzEBq9GMuvc9zMt4H
2024-07-10T16:23:44.506456+00:00 - INFO Adding block to witness tree mainnet (length 339996): 3NLnokmmHBqS1j1R72QmgfX9PuJQwkUXyYJx6MQvfSAQsd1WuypT
2024-07-10T16:23:44.524075+00:00 - INFO Adding block to witness tree mainnet (length 339997): 3NKM9jEEvjJYfu6NTwWiCBKmrDu8fBdZ9qWGAK6Kt2EyGd9S7rP6
2024-07-10T16:23:44.543388+00:00 - INFO Adding block to witness tree mainnet (length 339997): 3NKNZrRm6HJHBCoKgJMhm6YUVmiad3Gsttw4mSkMs9erTb2hJ2aU
2024-07-10T16:23:44.548243+00:00 - INFO Adding block to witness tree mainnet (length 339998): 3NLR3iLDkFCsEmXH3DKnb4CtsiZSjuJADbZJRubczPW7Zy6swZhm
2024-07-10T16:23:44.551718+00:00 - INFO Adding block to witness tree mainnet (length 339998): 3NLU7iDTsBPKBsw4TSECMyug4qx399xrk63sbqPNsnRtDzrP6ffq
2024-07-10T16:23:44.555501+00:00 - INFO Adding block to witness tree mainnet (length 339998): 3NLqzDezd2TgHbSJa5TEumwV5gdR9pvumUy1AfQZv3pWjxg6p1hi
2024-07-10T16:23:44.561149+00:00 - INFO Adding block to witness tree mainnet (length 339999): 3NKKNbGRYmJxoC78d1N8Y2kQo1kfbx3s25pwusqDvcKpctXpH4Jz
2024-07-10T16:23:44.580846+00:00 - INFO Adding block to witness tree mainnet (length 340000): 3NKDuaNDD4UFLBXkLS5iJQvwrJh2wxehr1PGjGRC2vs6Crt7AuC2
2024-07-10T16:23:44.602281+00:00 - INFO Adding block to witness tree mainnet (length 340000): 3NKoRz5SoWqnEHEShUEdquozjs7dHxc4VtVF529XBSjyuvc1xA2z
2024-07-10T16:23:44.611721+00:00 - INFO Adding block to witness tree mainnet (length 340000): 3NLBuMSMEbKoESNz3hFzr7GHAgpRwak62LgNAM1HsySL5qH1FWG6
2024-07-10T16:23:44.616505+00:00 - INFO Adding block to witness tree mainnet (length 340000): 3NLobPkJXXAbqFWeRtykfundvWZKKr7BxvFHPSmB1WWGKbmXmAdf
2024-07-10T16:23:44.636066+00:00 - INFO Adding block to witness tree mainnet (length 340000): 3NLpTXZM8D6cpbc4H9dyRN5u6h8GyHS6su1MmzxYfGTotGMo9pLd
2024-07-10T16:23:44.636286+00:00 - INFO Ingested and applied 827296 blocks (0 B) to the witness tree in 364.979824ms
2024-07-10T16:23:44.705067+00:00 - INFO Creating Unix domain socket server at "/mnt/mina-indexer-prod/mina-indexer-2033610c.socket"
2024-07-10T16:23:44.705128+00:00 - INFO Created Unix domain socket server at "/mnt/mina-indexer-prod/mina-indexer-2033610c.socket"
2024-07-10T16:23:44.764658+00:00 - INFO Watching for new blocks in directory: /mnt/mina-indexer-prod/blocks-340000
2024-07-10T16:23:44.764752+00:00 - INFO Watching for staking ledgers in directory: /mnt/mina-indexer-prod/staking-ledgers

[jhult]

Creating a new mina indexer database... == ~6s.

Looking for witness tree root block == ~7s

Witness tree root block (length 339990)... takes ~7s between this and Adding new blocks from...

[trevorbernard]

Why is this a problem? I think 20 seconds is reasonable

[robinbb]

It's unnecessary. It's spending many seconds calculating things that should simply be stored persistently, and their state retrieved.

This is not urgent.

[trevorbernard]

You should include that in the description then