`Error getting container metrics: json: unsupported value: NaN` and have some high CPU usage spikes with COolify

[MatteoGauthier]

On my coolify instance, i've enabled Sentinel and I noticed high CPU usages spikes and lots of error logs for the sentinel container

[livghit]

Hey , I've inspected the code a bit and noticed this one happen in the getOneContainerMetrics func

func getOneContainerMetrics(containerID string, csv bool) (string, error) {
    ctx := context.Background()
    apiClient, err := client.NewClientWithOpts()
    if err != nil {
        return "", err
    }
    apiClient.NegotiateAPIVersion(ctx)
    defer apiClient.Close()
    metrics := ContainerMetrics{
        CPUUsagePercentage:    0,
        MemoryUsagePercentage: 0,
        MemoryUsed:            0,
        MemoryAvailable:       0,
        NetworkUsage:          NetworkDevice{},
    }
    container, err := apiClient.ContainerInspect(ctx, containerID)
    if err != nil {
        return "", err
    }
    stats, err := apiClient.ContainerStats(ctx, container.ID, false)
    if err != nil {
        return "", err
    }
    var v types.StatsJSON
    dec := json.NewDecoder(stats.Body)
    if err := dec.Decode(&v); err != nil {
        if err != io.EOF {
            fmt.Printf("Error decoding container stats: %v\n", err)
        }
    }
    defer stats.Body.Close()
    network_devices := v.Networks
    for _, device := range network_devices {
        metrics.NetworkUsage = NetworkDevice{
            Name:    device.InstanceID,
            RxBytes: device.RxBytes,
            TxBytes: device.TxBytes,
        }
    }

    metrics = ContainerMetrics{
        Time:                  getUnixTimeInMilliUTC(),
        CPUUsagePercentage:    calculateCPUPercent(v),
        MemoryUsagePercentage: calculateMemoryPercent(v),
        MemoryUsed:            calculateMemoryUsed(v),
        MemoryAvailable:       v.MemoryStats.Limit,
        NetworkUsage:          metrics.NetworkUsage,
    }
    jsonData, err := json.MarshalIndent(metrics, "", "    ")
    if err != nil {
        return "", err
    }
    if csv {
        return fmt.Sprintf("%s,%f,%d,%f\n", metrics.Time, metrics.CPUUsagePercentage, metrics.MemoryUsed, metrics.MemoryUsagePercentage), nil
    }
    return string(jsonData), nil
}

I think one of the calculation may be the reason this happen , but I am not sure .

metrics = ContainerMetrics{
        Time:                  getUnixTimeInMilliUTC(),
        CPUUsagePercentage:    calculateCPUPercent(v),
        MemoryUsagePercentage: calculateMemoryPercent(v),
        MemoryUsed:            calculateMemoryUsed(v),
        MemoryAvailable:       v.MemoryStats.Limit,
        NetworkUsage:          metrics.NetworkUsage,
    }

[livghit]

Tested the whole thing locally and inside docker . I wasn't able to reproduce you're error .... 🥲

[mutonby]

I have the same error, can it be deactivated or something?

[Rhiz3K]

Same today after 307 update, per message through discord disabling metrics helped

[andrasbacsai]

In coolify v312, I disabled Sentinel on all servers until this bug (and a few others) are not fixed.