Fail to connect to Mongos on Sharded cluster in K8s

gsanchezgavier commented 2 years ago

When monitoring a sharded clusters in K8s the integration fails to discover the mongos endpoint and no MongosSample is generated.

Description

Mongos metrics collection endpoints are discovered in the integration but in K8s the Id on collections.config.mongos contains the Pod name which cannot be resolved by the integration.

Expected Behavior

Support native mongos autodiscovery for K8s environments

Steps to Reproduce

Using bitnami mongo-sharded chart and configured the integration with discovery to the mongos pod

nri-mongodb:
        discovery:
          command:
            exec: /var/db/newrelic-infra/nri-discovery-kubernetes --port 10250 --tls
            match:
              label.app.kubernetes.io/component: mongos
        integrations:
          - name: nri-mongodb
            env:
              HOST: ${discovery.ip}
              ...

The integrations prints the error like: [ERR] Failed to connect to mongos server <mongos-pod-name>:27017: no reachable servers

Your Environment

Additional context

I workaround this by using creating a service in the Agent namespace that points to the mongos instance and which have the same name of the mongos pod (using StatefulSet kind to deploy mongos to have predictable pod names). But this was just for the propose of validate the issue.

roobre commented 2 years ago

Bitnami chart (and many others that deploy statefulSets) has a flag to deploy a service per replica, that could help with the DNS problem.

While there is currently no way to perform discovery on services, I think the name for this service should be very similar to the pod name (assuming it is deployed as a statefulSet). Perhaps we could use this to work around the problem.

mangulonr commented 1 year ago

Hi

The new MongoDB OHI has been released in public preview, this version will be GA during June.

Some of the new features provided:

Compatibility with MongoDB Atlas.
Support of the latest major MongoDB versions 5 and 6.
Compatibility with Percona Server for MongoDB.
Support of SCRAM authentication.
Integration with the entity explorer thanks to 3 new entity types (MongoDB instance, database, and collection).
Improved new MongoDB quickstart.
Improved metrics filtering capabilities.

For more information check the following resources.

Because it is based on Prometheus the metric names are different from our legacy OHI, check our migration guide if you need more details. Our legacy MongoDB OHI will be deprecated on FY24H2 (final date pending to be decided).

newrelic / nri-mongodb