getting error - unmarshal object while running the couchdb exporter

[kunalpalan]

Hi Gesellix,

I'm running the exporter using the below command couchdb-prometheus-exporter --config=/etc/couchdb-exporter/config.ini

E0507 13:52:47.770990 1215732 exporter.go:118] error collecting couchdb stats: json: cannot unmarshal object into Go struct field NodeInfo.couchdb of type string

screencapture-192-168-57-250-9984-metrics-2024-05-07-14_32_53.pdf

[gesellix]

Can you provide some more details about the CouchDB version you are using and whether it runs as single node or in a cluster?

[kunalpalan]

Hi , we are using "couchdb":"Welcome","version":"3.3.2" and it runs as single node

[gesellix]

@kunalpalan can you please check the couchdb info like shown below?

curl http://localhost:5984/

For me, the reponse looks like this:

{"couchdb":"Welcome","version":"3.3.2","git_sha":"11a234070","uuid":"8d375d33af396fd8d731de6df57a8e88","features":["access-ready","partitioned","pluggable-storage-engines","reshard","scheduler"],"vendor":{"name":"The Apache Software Foundation"}}

Does your CouchDB instance run in a container? Does the couchdb-prometheus-exporter run in a container, or by using a pre-built binary, or did you build/run it yourself?

I tried to reproduce the issue with the following setup:

docker swarm init
docker network create couchdb
docker run --rm -d --network couchdb --name couchdb -p 5984:5984 --env COUCHDB_USER=admin --env COUCHDB_PASSWORD=password couchdb:3.3.2
curl -X PUT --user admin:password http://localhost:5984/_users
docker run --rm -d --network couchdb --name couchdb-exporter -p 9984:9984 gesellix/couchdb-prometheus-exporter:v30.9.5 --couchdb.uri http://couchdb:5984 --couchdb.username admin --couchdb.password password
curl -s http://localhost:9984/metrics | grep couchdb_server_node_info
docker logs couchdb
docker logs couchdb-exporter

Cleanup of the setup above:

docker stop couchdb-exporter
docker stop couchdb
docker network rm couchdb

[kunalpalan]

Hi I used the pre-build binary not a container. I used the below curl and got the output. curl http://localhost:6984/

{"couchdb":"Welcome","version":"3.3.2","git_sha":"11a234070","uuid":"8ffb60e6bf8bbe82394c4f65adb7ceb8","features":["search","access-ready","partitioned","pluggable-storage-engines","reshard","scheduler"],"vendor":{"name":"The Apache Software Foundation"}}

[gesellix]

Can you share the couchdb-prometheus-exporter config, without the actual credentials? Also, do the exporter logs show any errors when you perform requests on the exporter's /metrics endpoint?

[kunalpalan]

Hi, Pls find the below config.ini content

couchdb.username=admin couchdb.password=example couchdb.uri=http://192.168.10.10:6984/_node/_local/_stats telemetry.address=192.168.10.10:9984 databases=_all_dbs scrape.interval=3s

[gesellix]

Please change the couchdb.uri to ~couchdb.uri=http://192.168.10.10.6984~ couchdb.uri=http://192.168.10.10:6984. The couchdb-exporter will then navigate to the _stats and other required endpoints by itself.

[kunalpalan]

1. I have changed couchdb.uri=http://192.168.10.10.6984 , but i get below error

E0523 15:35:39.231336 3321696 exporter.go:118] Get "http://192.168.10.10.6984/_all_dbs": dial tcp: lookup 192.168.10.10.6984: no such host

2. I have changed couchdb.uri=http://192.168.10.10:6984, but i get below error

[root@dchxdqcouch1 couchdb-exporter]# couchdb-prometheus-exporter --config=/etc/couchdb-exporter/config.ini I0523 15:35:59.126794 3321712 couchdb-exporter.go:357] adopted logging config: {toStderr:true alsoToStderr:false verbosity:0 stderrThreshold:2 logDir:} I0523 15:35:59.127275 3321712 exporter.go:109] Asynchronously scraping the CouchDB stats at an interval of 3s I0523 15:35:59.127707 3321712 couchdb-exporter.go:296] Starting exporter version 30.9.5 at '192.168.10.10:9984' to read from CouchDB at 'http://192.168.10.10:6984' I0523 15:35:59.128005 3321712 klog_logger.go:20] "-" level="info" msg="Listening on" address="192.168.10.10:9984" I0523 15:35:59.128039 3321712 klog_logger.go:20] "-" level="info" msg="TLS is disabled." http2=false address="192.168.10.10:9984" E0523 15:36:08.506508 3321712 exporter.go:118] error collecting couchdb stats: error reading database 'pfm_3_1046885_audit' stats: status 500 Internal Server Error (500): {"error":"error","reason":"all_dbs_active"} E0523 15:38:21.343843 3321712 exporter.go:118] error collecting couchdb stats: error reading database 'pfm_3_1005206_audit' stats: status 500 Internal Server Error (500): {"error":"error","reason":"all_dbs_active"} E0523 15:40:33.062100 3321712 exporter.go:118] Get "http://192.168.10.10:6984/_all_dbs": dial tcp 192.168.10.10:6984: connect: connection timed out E0523 15:41:05.251134 3321712 exporter.go:118] error collecting couchdb stats: error reading database 'pfm_3_1017305_audit' stats: status 500 Internal Server Error (500): {"error":"internal_server_error","reason":"No DB shards could be opened.","ref":3085039011}

[gesellix]

The error all_dbs_active is interesting. Did you observe high load on the CouchDB host or can you check for general availability of the CouchDB for read and write operations? The exporter should retry regularly at the logged interval of 3s. If this would put too much load on your CouchDB host, you should decrease the scrape frequency.

[kunalpalan]

There is less load in the server.

[gesellix]

I'm not sure if this is related to https://github.com/apache/couchdb/issues/4076.

Looking at the supported features of your CouchDB instance, I stumbled over the "search" feature. Can you tell if your instance is running with a plain/default setup or with custom/additional features?

[kunalpalan]

config.ini file for reference databases=_testdb1 - working fine databases=_all_dbs - getting error

I'm getting below error, when i mention all databases. If I mention any one db, It's working fine.

E0902 17:33:08.482643 584577 collector.go:446] Error collecting stats: error collecting couchdb stats: error reading database 'all_dbs' stats: status 404 Object Not Found (404): {"error":"not_found","reason":"Database does not exist."} E0902 17:33:08.616529 584577 collector.go:446] Error collecting stats: error collecting couchdb stats: error reading database 'all_dbs' stats: status 404 Object Not Found (404): {"error":"not_found","reason":"Database does not exist."} E0902 17:33:10.477424 584577 collector.go:446] Error collecting stats: error collecting couchdb stats: error reading database 'all_dbs' stats: status 404 Object Not Found (404): {"error":"not_found","reason":"Database does not exist."}

And is there any way to view Search plugin status(up/down) and Search index running status

[kunalpalan]

Any update on this