Trying to collect mongodb metrics in a sharded mongodb deployment. As the documentation stats - "For a sharded MongoDB deployment, please specify a list of the mongos hosts." But when providing mongos as a host the receiver throws an error :
Oct 22 05:59:56 ip-10-0-31-77 otelcol-sumo[5498]: 2024-10-22T05:59:56.723Z error scraperhelper/scrapercontroller.go:197 Error scraping metrics {"kind": "receiver", "name": "mongodb/localhost", "data_type": "metrics", "error": "failed to find storage engine for cache operations; failed to collect metric mongodb.global_lock.time: could not find key for metric; failed to collect metric mongodb.operation.repl.count with attribute(s) delete: could not find key for metric; failed to collect metric mongodb.operation.repl.count with attribute(s) getmore: could not find key for metric; failed to collect metric mongodb.operation.repl.count with attribute(s) command: could not find key for metric; failed to collect metric mongodb.operation.repl.count with attribute(s) insert: could not find key for metric; failed to collect metric mongodb.operation.repl.count with attribute(s) query: could not find key for metric; failed to collect metric mongodb.operation.repl.count with attribute(s) update: could not find key for metric; failed to find storage engine for session count; failed to fetch top stats metrics: (CommandNotFound) no such cmd: top; failed to collect metric mongodb.document.operation.count with attribute(s) inserted, admin: could not find key for metric; failed to collect metric mongodb.document.operation.count with attribute(s) updated, admin: could not find key for metric; failed to collect metric mongodb.document.operation.count with attribute(s) deleted, admin: could not find key for metric; failed to collect metric mongodb.document.operation.count with attribute(s) inserted, config: could not find key for metric; failed to collect metric mongodb.document.operation.count with attribute(s) updated, config: could not find key for metric; failed to collect metric mongodb.document.operation.count with attribute(s) deleted, config: could not find key for metric", "scraper": "mongodb"}
The error occurs because there are certain command which are used by receiver for scrapping metrics like :
db.runCommand(
{
top: 1
}
)
which do not work on mongos but do work on mongod.
Also there are certain metrics which are not part of the output for commands on mongos like - "mongodb.global_lock.time"
Is this a limitation of the receiver. Since the receiver works properly when run against mongod instance. This contradicts the documentation (since it says we can give mongos instance).
Steps to Reproduce
Create a mongodb sharded infra following the steps here
Install OpenTelemetry collector on instance with mongos and provide hostname as mongos address.
Expected Result
All metrics should be collected successfully.
Actual Result
Collector throws error in collecting certain metrics. Error log in description
Component(s)
receiver/mongodb
What happened?
Description
Trying to collect mongodb metrics in a sharded mongodb deployment. As the documentation stats - "For a sharded MongoDB deployment, please specify a list of the mongos hosts." But when providing mongos as a host the receiver throws an error :
Oct 22 05:59:56 ip-10-0-31-77 otelcol-sumo[5498]: 2024-10-22T05:59:56.723Z error scraperhelper/scrapercontroller.go:197 Error scraping metrics {"kind": "receiver", "name": "mongodb/localhost", "data_type": "metrics", "error": "failed to find storage engine for cache operations; failed to collect metric mongodb.global_lock.time: could not find key for metric; failed to collect metric mongodb.operation.repl.count with attribute(s) delete: could not find key for metric; failed to collect metric mongodb.operation.repl.count with attribute(s) getmore: could not find key for metric; failed to collect metric mongodb.operation.repl.count with attribute(s) command: could not find key for metric; failed to collect metric mongodb.operation.repl.count with attribute(s) insert: could not find key for metric; failed to collect metric mongodb.operation.repl.count with attribute(s) query: could not find key for metric; failed to collect metric mongodb.operation.repl.count with attribute(s) update: could not find key for metric; failed to find storage engine for session count; failed to fetch top stats metrics: (CommandNotFound) no such cmd: top; failed to collect metric mongodb.document.operation.count with attribute(s) inserted, admin: could not find key for metric; failed to collect metric mongodb.document.operation.count with attribute(s) updated, admin: could not find key for metric; failed to collect metric mongodb.document.operation.count with attribute(s) deleted, admin: could not find key for metric; failed to collect metric mongodb.document.operation.count with attribute(s) inserted, config: could not find key for metric; failed to collect metric mongodb.document.operation.count with attribute(s) updated, config: could not find key for metric; failed to collect metric mongodb.document.operation.count with attribute(s) deleted, config: could not find key for metric", "scraper": "mongodb"}
The error occurs because there are certain command which are used by receiver for scrapping metrics like :
which do not work on mongos but do work on mongod.
Also there are certain metrics which are not part of the output for commands on mongos like - "mongodb.global_lock.time" Is this a limitation of the receiver. Since the receiver works properly when run against mongod instance. This contradicts the documentation (since it says we can give mongos instance).
Steps to Reproduce
Create a mongodb sharded infra following the steps here Install OpenTelemetry collector on instance with mongos and provide hostname as mongos address.
Expected Result
All metrics should be collected successfully.
Actual Result
Collector throws error in collecting certain metrics. Error log in description
Collector version
v0.108.0
Environment information
Environment
OS: (e.g., "Ubuntu 22.04") Compiler(if manually compiled): (e.g., "go 14.2")
OpenTelemetry Collector configuration
No response
Log output
No response
Additional context
No response