search

peimanja / artifactory_exporter

JFrog Artifactory Prometheus Exporter written in Go
Apache License 2.0
141 stars 37 forks source link

Empty metrics for artifactory_storage_filestore_free_bytes and artifactory_storage_filestore_used_bytes #21

Closed mfominov closed 4 years ago

mfominov commented 4 years ago

When i'm trying to calculate file storage % usage, i've faced an issue with empty metrics artifactory_storage_filestore_free_bytes and artifactory_storage_filestore_used_bytes

Overview of the Issue

A paragraph or two about the issue you're experiencing.

Reproduction Steps

Steps to reproduce this issue:

Operating system and Environment details

i'm running peimanja/artifactory_exporter:v0.4.1 artifactory version 6.15.0

Logs

In artifactory exporter i see only start message in log level=info ts=2020-01-27T14:32:04.457Z caller=main.go:30 msg="Listening on address" address=:9531

mfominov commented 4 years ago

maybe problem in format of used/free space, this is not only size by percentage too. "fileStoreSummary": { "storageType": "file-system", "storageDirectory": "/opt/jfrog/artifactory/data/filestore", "totalSpace": "1.92 TB", "usedSpace": "948.65 GB (48.36%)", "freeSpace": "1,012.87 GB (51.64%)" },

peimanja commented 4 years ago

Sorry for the late reply. I will do some investigation this week and maybe add some debug log in a new minor version.

peimanja commented 4 years ago

maybe problem in format of used/free space, this is not only size by percentage too. "fileStoreSummary": { "storageType": "file-system", "storageDirectory": "/opt/jfrog/artifactory/data/filestore", "totalSpace": "1.92 TB", "usedSpace": "948.65 GB (48.36%)", "freeSpace": "1,012.87 GB (51.64%)" },

It supposed to handle those. Can you please check artifactory_exporter_json_parse_failures metric and see if it has a non-zero value?

mfominov commented 4 years ago

This metric has zero value(

peimanja commented 4 years ago

Please give v0.4.2 a try and pass --log.level=debug flag. Then look for debug logs like this:

level=debug ts=2020-02-08T08:36:09.126Z caller=collector.go:151 msg="Fetching http" path=https://artifactory.lululemoninternal.com/artifactory/api/storageinfo
level=debug ts=2020-02-08T08:36:09.350Z caller=storage.go:103 msg="Converting size to bytes"
level=debug ts=2020-02-08T08:36:09.350Z caller=storage.go:85 msg="Removing other characters to extract number from string"
level=debug ts=2020-02-08T08:36:09.350Z caller=storage.go:94 msg="Successfully converted string to number" string="8,388,607.67 TB (100%)" number=8.388607671e+06
level=debug ts=2020-02-08T08:36:09.350Z caller=storage.go:122 msg="Successfully converted string to bytes" string="8,388,607.67 TB (100%)" value=9.22337167511545e+18
level=debug ts=2020-02-08T08:36:09.350Z caller=storage.go:152 msg="Registering metric" metric=filestoreFree type=cluster-s3 directory=/var/opt/jfrog/artifactory-ha/cache value=9.22337167511545e+18
level=debug ts=2020-02-08T08:36:09.350Z caller=storage.go:85 msg="Removing other characters to extract number from string"
level=debug ts=2020-02-08T08:36:09.350Z caller=storage.go:94 msg="Successfully converted string to number" string=26,486 number=26486
level=debug ts=2020-02-08T08:36:09.350Z caller=storage.go:132 msg="Registering metric" metric=binaries value=26486
level=debug ts=2020-02-08T08:36:09.350Z caller=storage.go:103 msg="Converting size to bytes"
level=debug ts=2020-02-08T08:36:09.350Z caller=storage.go:85 msg="Removing other characters to extract number from string"
level=debug ts=2020-02-08T08:36:09.350Z caller=storage.go:94 msg="Successfully converted string to number" string="176.62 GB" number=176.62
level=debug ts=2020-02-08T08:36:09.350Z caller=storage.go:122 msg="Successfully converted string to bytes" string="176.62 GB" value=1.8964428095488e+11
level=debug ts=2020-02-08T08:36:09.350Z caller=storage.go:142 msg="Registering metric" metric=binariesSize value=1.8964428095488e+11
level=debug ts=2020-02-08T08:36:09.350Z caller=storage.go:103 msg="Converting size to bytes"
level=debug ts=2020-02-08T08:36:09.350Z caller=storage.go:85 msg="Removing other characters to extract number from string"
level=debug ts=2020-02-08T08:36:09.350Z caller=storage.go:94 msg="Successfully converted string to number" string="8,388,608.00 TB" number=8.388608e+06
level=debug ts=2020-02-08T08:36:09.350Z caller=storage.go:122 msg="Successfully converted string to bytes" string="8,388,608.00 TB" value=9.223372036854776e+18
level=debug ts=2020-02-08T08:36:09.350Z caller=storage.go:152 msg="Registering metric" metric=filestore type=cluster-s3 directory=/var/opt/jfrog/artifactory-ha/cache value=9.223372036854776e+18
level=debug ts=2020-02-08T08:36:09.350Z caller=storage.go:103 msg="Converting size to bytes"
level=debug ts=2020-02-08T08:36:09.350Z caller=storage.go:85 msg="Removing other characters to extract number from string"
level=debug ts=2020-02-08T08:36:09.350Z caller=storage.go:94 msg="Successfully converted string to number" string="338.99 GB (0%)" number=338.99
level=debug ts=2020-02-08T08:36:09.350Z caller=storage.go:122 msg="Successfully converted string to bytes" string="338.99 GB (0%)" value=3.6398774091776e+11
level=debug ts=2020-02-08T08:36:09.350Z caller=storage.go:152 msg="Registering metric" metric=filestoreUsed type=cluster-s3 directory=/var/opt/jfrog/artifactory-ha/cache value=3.6398774091776e+11
level=debug ts=2020-02-08T08:36:09.350Z caller=storage.go:85 msg="Removing other characters to extract number from string"
level=debug ts=2020-02-08T08:36:09.350Z caller=storage.go:94 msg="Successfully converted string to number" string=31,712 number=31712
level=debug ts=2020-02-08T08:36:09.350Z caller=storage.go:132 msg="Registering metric" metric=artifacts value=31712
level=debug ts=2020-02-08T08:36:09.350Z caller=storage.go:103 msg="Converting size to bytes"
level=debug ts=2020-02-08T08:36:09.350Z caller=storage.go:85 msg="Removing other characters to extract number from string"
level=debug ts=2020-02-08T08:36:09.350Z caller=storage.go:94 msg="Successfully converted string to number" string="265.45 GB" number=265.45
level=debug ts=2020-02-08T08:36:09.350Z caller=storage.go:122 msg="Successfully converted string to bytes" string="265.45 GB" value=2.850247671808e+11
level=debug ts=2020-02-08T08:36:09.350Z caller=storage.go:142 msg="Registering metric" metric=artifactsSize value=2.850247671808e+11
mfominov commented 4 years ago

Some logs

level=debug ts=2020-02-10T08:18:27.634Z caller=storage.go:71 msg="Fetching storage info stats"
level=debug ts=2020-02-10T08:18:27.634Z caller=collector.go:151 msg="Fetching http" path=https://art.lmru.tech/artifactory/api/storageinfo
level=debug ts=2020-02-10T08:18:27.922Z caller=storage.go:85 msg="Removing other characters to extract number from string"
level=debug ts=2020-02-10T08:18:27.923Z caller=storage.go:94 msg="Successfully converted string to number" string=232,092 number=232092
level=debug ts=2020-02-10T08:18:27.923Z caller=storage.go:132 msg="Registering metric" metric=artifacts value=232092
level=debug ts=2020-02-10T08:18:27.923Z caller=storage.go:103 msg="Converting size to bytes"
level=debug ts=2020-02-10T08:18:27.923Z caller=storage.go:85 msg="Removing other characters to extract number from string"
level=debug ts=2020-02-10T08:18:27.923Z caller=storage.go:94 msg="Successfully converted string to number" string="2.39 TB" number=2.39
level=debug ts=2020-02-10T08:18:27.923Z caller=storage.go:122 msg="Successfully converted string to bytes" string="2.39 TB" value=2.62783279038464e+12
level=debug ts=2020-02-10T08:18:27.923Z caller=storage.go:142 msg="Registering metric" metric=artifactsSize value=2.62783279038464e+12
level=debug ts=2020-02-10T08:18:27.923Z caller=storage.go:85 msg="Removing other characters to extract number from string"
level=debug ts=2020-02-10T08:18:27.923Z caller=storage.go:94 msg="Successfully converted string to number" string=136,396 number=136396
level=debug ts=2020-02-10T08:18:27.923Z caller=storage.go:132 msg="Registering metric" metric=binaries value=136396
level=debug ts=2020-02-10T08:18:27.923Z caller=storage.go:103 msg="Converting size to bytes"
level=debug ts=2020-02-10T08:18:27.923Z caller=storage.go:85 msg="Removing other characters to extract number from string"
level=debug ts=2020-02-10T08:18:27.923Z caller=storage.go:94 msg="Successfully converted string to number" string="1.12 TB" number=1.12
level=debug ts=2020-02-10T08:18:27.923Z caller=storage.go:122 msg="Successfully converted string to bytes" string="1.12 TB" value=1.23145302310912e+12
level=debug ts=2020-02-10T08:18:27.923Z caller=storage.go:142 msg="Registering metric" metric=binariesSize value=1.23145302310912e+12
level=debug ts=2020-02-10T08:18:27.923Z caller=storage.go:103 msg="Converting size to bytes"
level=debug ts=2020-02-10T08:18:27.923Z caller=storage.go:85 msg="Removing other characters to extract number from string"
level=debug ts=2020-02-10T08:18:27.923Z caller=storage.go:94 msg="Successfully converted string to number" string="1.92 TB" number=1.92
level=debug ts=2020-02-10T08:18:27.923Z caller=storage.go:122 msg="Successfully converted string to bytes" string="1.92 TB" value=2.11106232532992e+12
level=debug ts=2020-02-10T08:18:27.923Z caller=storage.go:152 msg="Registering metric" metric=filestore type=file-system directory=/opt/jfrog/artifactory/data/filestore value=2.11106232532992e+12
level=debug ts=2020-02-10T08:18:27.923Z caller=storage.go:103 msg="Converting size to bytes"
level=debug ts=2020-02-10T08:18:27.923Z caller=storage.go:85 msg="Removing other characters to extract number from string"
level=debug ts=2020-02-10T08:18:27.923Z caller=storage.go:152 msg="Registering metric" metric=filestoreUsed type=file-system directory=/opt/jfrog/artifactory/data/filestore value=0
level=debug ts=2020-02-10T08:18:27.923Z caller=storage.go:103 msg="Converting size to bytes"
level=debug ts=2020-02-10T08:18:27.923Z caller=storage.go:85 msg="Removing other characters to extract number from string"
level=debug ts=2020-02-10T08:18:27.923Z caller=storage.go:152 msg="Registering metric" metric=filestoreFree type=file-system directory=/opt/jfrog/artifactory/data/filestore value=0
level=debug ts=2020-02-10T08:18:27.923Z caller=storage.go:171 msg="Extracting repo summeriest"
level=debug ts=2020-02-10T08:18:27.923Z caller=storage.go:103 msg="Converting size to bytes"
peimanja commented 4 years ago

Thanks for the logs. Please let me know if v0.4.3 fixed the issue

mfominov commented 4 years ago

It work's)