minio reports total disk storage incorrectly on prometheus scraping API

[m-motawea]

using prometheus to monitor deployed minio solution on the grid. it is deployed with 2 data nodes and 1 parity each with 3 GB storage and one minio container but reports total disk space as 3.2 GB

needed for https://github.com/threefoldtech/home/issues/949 to trigger s3 auto-topup

Expected Behavior

should report sum of data nodes 6 GB

Current Behavior

it reports total disk storage around 3.2 GB as well as total

(js-sdk-IZdTbuKY-py3.6) maged@maged-Inspiron-3576:~/prometheus-2.22.1.linux-amd64$ curl http://172.29.3.2:9000/minio/prometheus/metrics -H "name: Authorization" -H "content: Bearer eyJhbGciOiJIUzUxMiIsInR5cCI6Ik
pXVCJ9.eyJleHAiOjQ3NTkwNDQ4MzYsImlzcyI6InByb21ldGhldXMiLCJzdWIiOiJtYWdpZGVudGZpbmFsIn0._ZWY5gOdF6sNTpaB7fZK8pL109i56JgNCPJ3ioAzpobM59LmMcE6xDPZdV20u3vjDlH-cPVyXrUdu0THc2-oRw"                                     
# HELP disk_storage_available Total available space left on the disk                                                                                                                                               
# TYPE disk_storage_available gauge                                                                                                                                                                                
disk_storage_available{disk=""} 3.221225472e+09                                                                                                                                                                    
# HELP disk_storage_total Total space on the disk
# TYPE disk_storage_total gauge
disk_storage_total{disk=""} 3.221225472e+09
# HELP disk_storage_used Total disk storage used on the disk
# TYPE disk_storage_used gauge
disk_storage_used{disk=""} 0

Your Environment

minio config

zosv2tst-4:~# cat /data/minio.yaml
password: ""
namespace: ""
datastor:
  shards:
  - address: '[2a02:1802:5e:0:74c8:d3ff:fe60:6887]:9900'
    namespace: 504409-1
    password: c71921d356004ea0a5511b335ddbec87
  - address: '[2a02:1802:5e:0:3cf6:9eff:fe82:9bd7]:9900'
    namespace: 504410-1
    password: c71921d356004ea0a5511b335ddbec87
  - address: '[2a02:1802:5e:0:8cb4:ebff:fe21:7745]:9900'
    namespace: 504411-1
    password: c71921d356004ea0a5511b335ddbec87
  spreading: random
  pipeline:
    block_size: 4194304
    hashing:
      type: blake2b_256
      private_key: ""
    compression:
      mode: ""
      type: snappy
    encryption:
      private_key: ""
      type: aes
    distribution:
      data_shards: 2
      parity_shards: 1
  tls:
    enabled: false
    server: ""
    root_ca: ""
    min_version: ""
    max_version: ""
jobs: 0
minio:
  healer:
    listen: ""

[muhamadazmy]

Just to update this issue with latest status as we discussed:

• diskstorage* metrics are reported by minio itself, and big chance they are totally wrong since minio does not know how our erasure coding is configured or working. so those values should not be used
• instead we need to use the miniozerostor* metrics, as we checked these value units seems to not be correct. we will check that to make sure reported values are in bytes.