Node Status page doesn't actually identify the broken grpc endpoint

[Valentine1898]

https://grpc.testnet.penumbra.zone endpoint is actually broken, but the Node Status page does not display this Instead, the page displays always Caught Up as true, while Latest Block Height displays random old blocks instead of new ones

https://github.com/penumbra-zone/web/assets/25391690/37983a55-c539-4bec-b12d-ad4148ccc5df

[grod220]

Oh whoa, the random latest block height is quite a strange behavior. From the client side of things, how could we determine there is an issue if the node is sending back random values for latestBlockHeight? Aka, what's a programmatic way we can determine the node is broken?

[Valentine1898]

From the client side of things, how could we determine there is an issue if the node is sending back random values for latestBlockHeight?

I suspect that there is no way to determine it

But maybe it makes sense to extend the functionality of Node Status page and check not only TendermintProxyService

The idea is that we can execute different grpc queries that do not require passing parameters, such as AppParameters, or queries where we can pass a known value as a parameter, such as a CompactBlock

We could display the availability of services like this

Service Name	Status
CurrentGasPrices	✅
AppParameters	✅
CompactBlock	❌

[Valentine1898]

Aka, what's a programmatic way we can determine the node is broken?

Theoretically, we could add a state, write there the latestBlockHeight value obtained from the node, and at the next request check that the new latestBlockHeight is greater than or equal to the value stored in the state. But personally, I don't think we should do it just to determine this strange behavior