Seeing issues in resource starved instances when CB_RESULTSET=1 the REST Endpoint will say there are 60 samples but ns_server will return between 53-58 samples, missing stat tickets indicating resource issues. However, when these situations arise we should still be returning stats.
The issue is that sample_list assumes there are 60 samples, this is not always the case, even in a normal working state ns_server does not guarantee 60 samples there is a +/- 1 factor here as well. We need a safety check when looping over stats that checks to see if the length of the stat list is less than the first position in the samples_list and if that is, output just the last stat value regardless of position.
This logic should apply to all REST endpoints from Couchbase Server, it should not apply to cbstats, process_exporter and node_exporter endpoints
Seeing issues in resource starved instances when
CB_RESULTSET=1the REST Endpoint will say there are60samples butns_serverwill return between 53-58 samples, missing stat tickets indicating resource issues. However, when these situations arise we should still be returning stats.The issue is that
sample_listassumes there are 60 samples, this is not always the case, even in a normal working statens_serverdoes not guarantee 60 samples there is a +/- 1 factor here as well. We need a safety check when looping over stats that checks to see if the length of the stat list is less than the first position in thesamples_listand if that is, output just the last stat value regardless of position.This logic should apply to all REST endpoints from Couchbase Server, it should not apply to cbstats, process_exporter and node_exporter endpoints