minio / madmin-go

The MinIO Admin Go Client SDK provides APIs to manage MinIO services
https://pkg.go.dev/github.com/minio/madmin-go/v3
GNU Affero General Public License v3.0
90 stars 63 forks source link

Get the specified metrics because all BucketMetrics are too large #275

Closed nowinkeyy closed 6 months ago

nowinkeyy commented 6 months ago

The metricsRespBodyLimit size is 10MB, but the actual data is 14MB>10MB, causing an error. Can I specify a metric to access? For example, minio_bucket_usage_total_bytes metric.

const (
    metricsRespBodyLimit = 10 << 20 // 10 MiB
)
// fetchMetrics - returns Metrics of given subsystem in Prometheus format
func (client *MetricsClient) fetchMetrics(ctx context.Context, subSystem string) ([]*prom2json.Family, error) {
    reqData := metricsRequestData{
        relativePath: "/v2/metrics/" + subSystem,
    }

    // Execute GET on /minio/v2/metrics/<subSys>
    resp, err := client.executeGetRequest(ctx, reqData)
    if err != nil {
        return nil, err
    }
    defer closeResponse(resp)

    if resp.StatusCode != http.StatusOK {
        return nil, httpRespToErrorResponse(resp)
    }

    return parsePrometheusResults(io.LimitReader(resp.Body, metricsRespBodyLimit))
}
nowinkeyy commented 6 months ago

I'm not sure what the 10MB limit means here. My current workaround is to copy some of the code and filter out the metric information before parsePrometheusResults. After filtering, the data goes from 14MB to 72KB so that it doesn't exceed 10MB.

harshavardhana commented 6 months ago

This is addressed in #276