open-telemetry / opentelemetry-collector-contrib

Contrib repository for the OpenTelemetry Collector
https://opentelemetry.io
Apache License 2.0
3.02k stars 2.33k forks source link

Mongo receiver throws error when scrapping metrics from mongos instance #35916

Open sumoanema opened 2 hours ago

sumoanema commented 2 hours ago

Component(s)

receiver/mongodb

What happened?

Description

Trying to collect mongodb metrics in a sharded mongodb deployment. As the documentation stats - "For a sharded MongoDB deployment, please specify a list of the mongos hosts." But when providing mongos as a host the receiver throws an error : Oct 22 05:59:56 ip-10-0-31-77 otelcol-sumo[5498]: 2024-10-22T05:59:56.723Z error scraperhelper/scrapercontroller.go:197 Error scraping metrics {"kind": "receiver", "name": "mongodb/localhost", "data_type": "metrics", "error": "failed to find storage engine for cache operations; failed to collect metric mongodb.global_lock.time: could not find key for metric; failed to collect metric mongodb.operation.repl.count with attribute(s) delete: could not find key for metric; failed to collect metric mongodb.operation.repl.count with attribute(s) getmore: could not find key for metric; failed to collect metric mongodb.operation.repl.count with attribute(s) command: could not find key for metric; failed to collect metric mongodb.operation.repl.count with attribute(s) insert: could not find key for metric; failed to collect metric mongodb.operation.repl.count with attribute(s) query: could not find key for metric; failed to collect metric mongodb.operation.repl.count with attribute(s) update: could not find key for metric; failed to find storage engine for session count; failed to fetch top stats metrics: (CommandNotFound) no such cmd: top; failed to collect metric mongodb.document.operation.count with attribute(s) inserted, admin: could not find key for metric; failed to collect metric mongodb.document.operation.count with attribute(s) updated, admin: could not find key for metric; failed to collect metric mongodb.document.operation.count with attribute(s) deleted, admin: could not find key for metric; failed to collect metric mongodb.document.operation.count with attribute(s) inserted, config: could not find key for metric; failed to collect metric mongodb.document.operation.count with attribute(s) updated, config: could not find key for metric; failed to collect metric mongodb.document.operation.count with attribute(s) deleted, config: could not find key for metric", "scraper": "mongodb"}

The error occurs because there are certain command which are used by receiver for scrapping metrics like :

db.runCommand(
   {
     top: 1
   }
)

which do not work on mongos but do work on mongod.

Also there are certain metrics which are not part of the output for commands on mongos like - "mongodb.global_lock.time" Is this a limitation of the receiver. Since the receiver works properly when run against mongod instance. This contradicts the documentation (since it says we can give mongos instance).

Steps to Reproduce

Create a mongodb sharded infra following the steps here Install OpenTelemetry collector on instance with mongos and provide hostname as mongos address.

Expected Result

All metrics should be collected successfully.

Actual Result

Collector throws error in collecting certain metrics. Error log in description

Collector version

v0.108.0

Environment information

Environment

OS: (e.g., "Ubuntu 22.04") Compiler(if manually compiled): (e.g., "go 14.2")

OpenTelemetry Collector configuration

No response

Log output

No response

Additional context

No response

github-actions[bot] commented 2 hours ago

Pinging code owners: