When new metrics are introduced in OpenSearch we do not learn about this when upgrading. There is simply no logic to automatically detect this and those new metric stay unnoticed.
To improve this situation we can implement a new set of tests that will pull JSON data containing metrics and count number of members of the JSON structure and compare to hard-coded numbers. This way we can easily learn that something new was added. Theoretically we can miss the case when one metric is removed and one added (the count will be the same) but this will be caught by other existing tests that "call specific metrics by name".
When new metrics are introduced in OpenSearch we do not learn about this when upgrading. There is simply no logic to automatically detect this and those new metric stay unnoticed.
To improve this situation we can implement a new set of tests that will pull JSON data containing metrics and count number of members of the JSON structure and compare to hard-coded numbers. This way we can easily learn that something new was added. Theoretically we can miss the case when one metric is removed and one added (the count will be the same) but this will be caught by other existing tests that "call specific metrics by name".