As of now, the healtcheck is just fetching the expected block number from cardanoexplorer which means that if cardanoexplorer goes down/migrates to something new, it will stop working until we update it. This indeed happened today (May 29) when IOHK apparently shut down the https://cardanoexplorer.com/api/blocks/pages endpoint as part of a migration to a newer API.
Relying on cardanoexplorer alone is unnecessarily risky and as a fallback, we should be checking if new blocks are appearing to mitigate the risk of unknowingly serving stale data for an extended period of time. Alternatively, we can check the timestamp at which the last block in the DB appeared (this is perhaps more robust because it will be able to disable the node right after starting), the risk being that if we set a threshold too strictly, it may disable the instance if a block legitimately fails to appear in the blockchain for several slots in a row.
Ideally, we should also notify or at least log if the call to cardanoexplorer fails
As of now, the healtcheck is just fetching the expected block number from cardanoexplorer which means that if cardanoexplorer goes down/migrates to something new, it will stop working until we update it. This indeed happened today (May 29) when IOHK apparently shut down the https://cardanoexplorer.com/api/blocks/pages endpoint as part of a migration to a newer API.
Relying on cardanoexplorer alone is unnecessarily risky and as a fallback, we should be checking if new blocks are appearing to mitigate the risk of unknowingly serving stale data for an extended period of time. Alternatively, we can check the timestamp at which the last block in the DB appeared (this is perhaps more robust because it will be able to disable the node right after starting), the risk being that if we set a threshold too strictly, it may disable the instance if a block legitimately fails to appear in the blockchain for several slots in a row.
Ideally, we should also notify or at least log if the call to cardanoexplorer fails