Alluxio / alluxio

Alluxio, data orchestration for analytics and machine learning in the cloud
https://www.alluxio.io
Apache License 2.0
6.87k stars 2.94k forks source link

Alluxio monitor bug #16099

Open flyhighzy opened 2 years ago

flyhighzy commented 2 years ago

Alluxio Version: 2.8.1

Describe the bug image As shown, "Bytes Read Cache" here is negative, which is obviously incorrect

To Reproduce After alluxio cluster is ready, at the beginning of job read data from alluxio, it always occurs.

Expected behavior show the right data

HelloHorizon commented 2 years ago

@beinan @dbw9580 can you take a look?

singer-bin commented 2 years ago

@flyhighzy Can you post your json template and see what indicator (bytes read cache) is?

flyhighzy commented 2 years ago

here is the panel json:

{
  "datasource": "$alluxio_datasource",
  "aliasColors": {},
  "bars": false,
  "dashLength": 10,
  "dashes": false,
  "fieldConfig": {
    "defaults": {
      "custom": {}
    },
    "overrides": []
  },
  "fill": 1,
  "fillGradient": 0,
  "gridPos": {
    "h": 8,
    "w": 6,
    "x": 12,
    "y": 10
  },
  "hiddenSeries": false,
  "id": 109,
  "legend": {
    "avg": false,
    "current": false,
    "hideEmpty": true,
    "max": false,
    "min": false,
    "show": true,
    "total": false,
    "values": false
  },
  "lines": true,
  "linewidth": 1,
  "nullPointMode": "null",
  "options": {
    "alertThreshold": true,
    "dataLinks": []
  },
  "percentage": false,
  "pluginVersion": "7.3.4",
  "pointradius": 2,
  "points": false,
  "renderer": "flot",
  "seriesOverrides": [],
  "spaceLength": 10,
  "stack": false,
  "steppedLine": false,
  "targets": [
    {
      "expr": "Cluster_BytesReadRemote{fluid_runtime=~\"$fluid_runtime\",namespace=~\"$namespace\"} - Cluster_BytesReadUfsAll{fluid_runtime=~\"$fluid_runtime\",namespace=~\"$namespace\"}",
      "interval": "",
      "legendFormat": "Bytes read cache",
      "refId": "A"
    }
  ],
  "thresholds": [],
  "timeFrom": null,
  "timeRegions": [],
  "timeShift": null,
  "title": "Bytes Read Cache",
  "tooltip": {
    "shared": true,
    "sort": 0,
    "value_type": "individual"
  },
  "type": "graph",
  "xaxis": {
    "buckets": null,
    "mode": "time",
    "name": null,
    "show": true,
    "values": []
  },
  "yaxes": [
    {
      "format": "bytes",
      "label": null,
      "logBase": 1,
      "max": null,
      "min": null,
      "show": true
    },
    {
      "format": "short",
      "label": null,
      "logBase": 1,
      "max": null,
      "min": null,
      "show": true
    }
  ],
  "yaxis": {
    "align": false,
    "alignLevel": null
  }
}
singer-bin commented 2 years ago

@flyhighzy Seems to be out of order. Cluster_BytesReadUfsAll - Cluster_BytesReadRemote

flyhighzy commented 2 years ago

But actually, as time goes on, "Bytes read cache" metric become positive and seems right

singer-bin commented 2 years ago

sorry i was wrong, it looks like a bug. @flyhighzy

singer-bin commented 2 years ago

@flyhighzy look it

singer-bin commented 2 years ago

image

singer-bin commented 2 years ago

image

github-actions[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in two weeks if no further activity occurs. Thank you for your contributions.