oxidecomputer / omicron

Omicron: Oxide control plane
Mozilla Public License 2.0
252 stars 40 forks source link

[clickhouse] WIP: Internal monitoring endpoints #7114

Open karencfv opened 2 days ago

karencfv commented 2 days ago

Queries per second

$ $ curl "http://[::1]:8888/timeseries/metric-log/ProfileEvent_Query?time_range=3600" | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  2883  100  2883    0     0   5912      0 --:--:-- --:--:-- --:--:--  5907
[
  {
    "time": "2024-11-20T04:06:00Z",
    "value": 0.0
  },
  {
    "time": "2024-11-20T04:07:00Z",
    "value": 0.0
  },
  {
    "time": "2024-11-20T04:08:00Z",
    "value": 0.0
  },
  {
    "time": "2024-11-20T04:09:00Z",
    "value": 0.0
  },
  {
    "time": "2024-11-20T04:10:00Z",
    "value": 0.0
  },
  {
    "time": "2024-11-20T04:11:00Z",
    "value": 0.06666666666666667
  },

Disk usage

$ $ curl "http://[::1]:8888/timeseries/async-metric-log/DiskUsed_default?interval=120&time_range=3600" | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1786  100  1786    0     0   3728      0 --:--:-- --:--:-- --:--:--  3728
[
  {
    "time": "2024-11-20T04:04:00Z",
    "value": 478592288085.3333
  },
  {
    "time": "2024-11-20T04:06:00Z",
    "value": 478626093841.06665
  },
  {
    "time": "2024-11-20T04:08:00Z",
    "value": 478695936853.3333
  },
  {
    "time": "2024-11-20T04:10:00Z",
    "value": 478623133969.06665
  },
  {
    "time": "2024-11-20T04:12:00Z",
    "value": 478688198178.13336
  },
  {
    "time": "2024-11-20T04:14:00Z",
    "value": 478791053653.3333
  },
karencfv commented 14 hours ago

Have been playing around with this. I think we can get some sort of Ok looking charts with this data:

Screenshot 2024-11-22 at 2 55 30 PM