[mongodb-shared] Container metrics can't start

[PhanSon95]

Which chart: Chart version: 1.0.8

Describe the bug I've deployed mongodb-sharded success to my cluster (include 3 nodes master) , all resources are running and ready. But inside POD include metrics container has error log

time="2020-11-07T05:09:24Z" level=info msg="Starting mongodb_exporter (version=0.10.0, branch=v0.10.0, revision=bf683745093a9210ebacbeb235bb792e21d17389)" source="mongodb_exporter.go:94"
time="2020-11-07T05:09:24Z" level=info msg="Build context (go=go1.12.9, user=travis@build.travis-ci.com, date=20190918-08:07:48)" source="mongodb_exporter.go:95"
time="2020-11-07T05:09:58Z" level=error msg="Could not get MongoDB BuildInfo: server selection error: server selection timeout\ncurrent topology: Type: Single\nServers:\nAddr: localhost:27017, Type: Unknown, State: Connected, Average RTT: 0, Last error: connection(localhost:27017[-121]) connection is closed\n!" source="connection.go:98"
time="2020-11-07T05:09:58Z" level=error msg="Problem gathering the mongo server version: server selection error: server selection timeout\ncurrent topology: Type: Single\nServers:\nAddr: localhost:27017, Type: Unknown, State: Connected, Average RTT: 0, Last error: connection(localhost:27017[-121]) connection is closed\n" source="mongodb_collector.go:203"
time="2020-11-07T05:09:58Z" level=info msg="Starting HTTP server for http://:9216/metrics ..." source="server.go:140"

To Reproduce Steps to reproduce the behavior:

1. Check log of container metrics of other POD include: kubectl logs <POD_NAME> -c metrics -n <NAMESPACE>
2. See error

Expected behavior Container metrics running whic will help me monitoring mongodb-shared better

Version of Helm and Kubernetes:

• Output of helm version:

version.BuildInfo{Version:"v3.2.3", GitCommit:"8f832046e258e2cb800894579b1b3b50c2d83492", GitTreeState:"clean", GoVersion:"go1.13.12"}

• Output of kubectl version:

Server Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.7", GitCommit:"b4455102ef392bf7d594ef96b97a4caa79d729d9", GitTreeState:"clean", BuildDate:"2020-06-17T11:32:20Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}

[carrodher]

Probably the error messages are shown in the first moment while the mongo nodes are not ready yet, then everything should be normal:

time="2020-11-07T05:09:58Z" level=info msg="Starting HTTP server for http://:9216/metrics ..." source="server.go:140"

Are you able to see/collect the metrics obviating the error messages in the log?

I access the container and execute the test command and it seems it's connected properly to mongodb even the error messages appeared at the beginning:

$ /bin/mongodb_exporter --mongodb.uri mongodb://root:7ztYhXXpbu@localhost:27017/admin --test
{
  "Version": "4.4.2",
  "VersionArray": [
    4,
    4,
    2,
    0
  ],
  "GitVersion": "15e73dc5738d2278b688f8929aee605fe4279b0e",
  "OpenSSLVersion": "",
  "SysInfo": "",
  "Bits": 64,
  "Debug": false,
  "MaxObjectSize": 16777216
}

that is using the same address than the process running in the container:

$ ps xufa
USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
1001           9  0.0  0.2 717988 19528 ?        Sl   13:04   0:01 /bin/mongodb_exporter --mongodb.uri mongodb://root:7ztYhXXpbu@localhost:27017/admin

[PhanSon95]

@carrodher That's all logs which container metrics logging, beside no more logs. Is that OK ? I accessed to metrics container and execute your command

I have no name!@mongodb-sharded-configsvr-0:/opt/bitnami/mongodb-exporter$ /bin/mongodb_exporter --mongodb.uri mongodb://root:mypassword@localhost:27017/admin --test
{
  "Version": "4.2.3",
  "VersionArray": [
    4,
    2,
    3,
    0
  ],
  "GitVersion": "6874650b362138df74be53d366bbefc321ea32d4",
  "OpenSSLVersion": "",
  "SysInfo": "",
  "Bits": 64,
  "Debug": false,
  "MaxObjectSize": 16777216
}

Same result with you. So it's Ok, right ? Thank you~

[PhanSon95]

@carrodher One more question. now I'm using kube-prometheus inside cluster. Question is: How can I confirm metric of mongo is using by prometheus ? Thanks you.

[carrodher]

To configure kube-prometheus you should enable the service monitor, you can take a look at the different customizable parameters related to metrics in https://github.com/bitnami/charts/tree/master/bitnami/mongodb-sharded#metrics-exporter, in the same way, there are some comments in the values.yaml section, for example, in relation to Prometheus, take a look at https://github.com/bitnami/charts/blob/17909ff576cc761464d791cc02bc2ad87f8354c2/bitnami/mongodb-sharded/values.yaml#L887

[PhanSon95]

@carrodher Yah, I got it. After enable serviceMonitor inside values.yaml, prometheus collected metrics, but I have a questions is: Why all pod of mongodb-sharded [configsrv,mongos,shard-data] also has metrics container ? But when using Grafana Dashboard with prometheus dataasource. I just see metrics from 3 instances of mongos, not all instance of mongo

[PhanSon95]

I'm using MongoDB Grafana Dashboard at here and bellow is setting of that dashboard

[carrodher]

Glad to see some advances. Which pods (or nodes) correspond to those IPs?

[PhanSon95]

@carrodher That is IPs of 3 POD manager by mongodb-sharded-mongos deployment. This is prometheus query to get mongodb connection for variable of dashboard settings Grafana

label_values(mongodb_connections, instance)

[rafariossaa]

Hi, @PhanSon95 I am not very sure of the current status of this issue, Did you finally got the data ?

[PhanSon95]

@rafariossaa Currently, the data which I saw on the Grafana Dashbard still have 3 instances of mongos. In my case deploy has:

• 6 shards = 18 POD sharded-data, 6 POD arbiter
• 3 POD configsrv
• 3 POD mongos and finally result that I want to be is all instances above. Could you give me your opinion about my case ? That's normal ? Thank you.

[CCJY]

Hello.

I got the same issue. The prometheus scraped metrics of mongos, but metrics of shardsvr and configsvr are dropped on prometheus. I don't know why metrics dropped except metrics of mongos. I want to get metrics of shardsvr and configsvr included mongodbmongod*.

Test

Everything seems okay.

kubectl exec --stdin --tty mongodb-sharded-shard1-data-0  -n mongodb-system -c metrics  -- /bin/bash

curl http://localhost:9216/metrics 

Service Discovery on Prometheus

Grafana

• Helm Version

  version.BuildInfo{Version:"v3.2.4", GitCommit:"0ad800ef43d3b826f31a5ad8dfbb4fe05d143688", GitTreeState:"clean", GoVersion:"go1.13.12"} 

• Kubectl Version

  Server Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.3+k3s3", GitCommit:"0e4fbfefe1dd8734756dfa4f9ab4fc89665cece4", GitTreeState:"clean", B
  uildDate:"2020-11-13T07:19:02Z", GoVersion:"go1.15.2", Compiler:"gc", Platform:"linux/amd64"}

[CCJY]

I got metrics of shardsvr and configsvr included mongodbmongod*. I can see metrics on grafana. Also, I referred to Additional Scrape Configuration.

[PhanSon95]

@CCJY Oh, As your refered here. Are we need set this

serviceMonitorSelector:
    matchLabels:
      team: frontend ???? (this is serviceMonitor which we want get)

[juan131]

Hi @CCJY @PhanSon95

Could you please share the values.yaml you used to fix the issue? I don't fully understand what modifications you did in the ServiceMonitor object to workaround it.

[PhanSon95]

@juan131 Here is my values.yaml. Could you help me check it ?

## Global Docker image parameters
## Please, note that this will override the image parameters, including dependencies, configured to use the global value
## Current available global Docker image parameters: imageRegistry, imagePullSecrets and storageClass
##
global: {}
#   imageRegistry: myRegistryName
#   imagePullSecrets:
#     - myRegistryKeySecretName
#   storageClass: myStorageClass

## Bitnami MongoDB Sharded image version
## ref: https://hub.docker.com/r/bitnami/mongodb-sharded/tags/
##
image:
  registry: docker.io
  repository: bitnami/mongodb-sharded
  tag: 4.4.2-debian-10-r0
  ## Specify a imagePullPolicy
  ## Defaults to 'Always' if image tag is 'latest', else set to 'IfNotPresent'
  ## ref: http://kubernetes.io/docs/user-guide/images/#pre-pulling-images
  ##
  pullPolicy: IfNotPresent
  ## Optionally specify an array of imagePullSecrets.
  ## Secrets must be manually created in the namespace.
  ## ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
  ##
  pullSecrets:

  ## Set to true if you would like to see extra information on logs
  ## It turns on Bitnami debugging in minideb-extras-base
  ## ref:  https://github.com/bitnami/minideb-extras-base
  ##
  debug: true

## String to partially override mongodb.fullname template (will maintain the release name)
##
nameOverride:

## String to fully override mongodb.fullname template
##
fullnameOverride:

## MongoDB credentials
##

## MongoDB root password
## If set to null it will be randomly generated
## ref: https://github.com/bitnami/bitnami-docker-mongodb/blob/master/README.md#setting-the-root-password-on-first-run
##
## mongodbRootPassword: password
##
mongodbRootPassword:

## Create a non-root MongoDB user
## ref: https://github.com/bitnami/bitnami-docker-mongodb/blob/master/README.md#setting-the-root-password-on-first-run
##
## mongodbUsername: user
##
mongodbUsername:

## non-root MongoDB user password
## ref: https://github.com/bitnami/bitnami-docker-mongodb/blob/master/README.md#setting-the-root-password-on-first-run
##
## mongodbPassword: password
##
mongodbPassword:

## Create database on first run
## ref: https://github.com/bitnami/bitnami-docker-mongodb/blob/master/README.md#setting-the-root-password-on-first-run
##
# mongodbDatabase: database
mongodbDatabase:

## Replica Set key (shared for shards and config servers)
## ref:
##
## replicaSetKey: testkey123
##
replicaSetKey:

## Name of a secret containing all the credentials above
## ref:
##
## existingSecret: name-of-existing-secret
##
existingSecret: mongodb-sharded

## Mount credentials as files instead of using environment variables
##
usePasswordFile: true

## Number of MongoDB Shards
## ref: https://docs.mongodb.com/manual/core/sharded-cluster-shards/
##
shards: 6

## Shard replica set properties
## ref: https://docs.mongodb.com/manual/replication/index.html
##
shardsvr:
  ## Properties for data nodes (primary and secondary)
  ##
  dataNode:
    ## Number of replicas. A value of replicas=1 is simply a primary node
    ##
    replicas: 3
    ## Configure resource requests and limits
    ## ref: http://kubernetes.io/docs/user-guide/compute-resources/
    ##
    resources: 
      requests:
          memory: "3Gi"
          cpu: "1"
      limits:
          memory: "7Gi"
          cpu: "2"
    ## MongoDB additional command line flags
    ##
    ## Can be used to specify command line flags, for example:
    ##
    ## mongodbExtraFlags:
    ##  - "--wiredTigerCacheSizeGB=2"
    ##
    mongodbExtraFlags:
      - --wiredTigerCacheSizeGB=1
    ## Pod priority
    ## https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/
    ##
    priorityClassName:
    ## Node selector
    ## ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselector
    ##
    nodeSelector: {}
    ## Affinity
    ## ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity
    ##
    affinity: {}
    ## Tolerations
    ## ref: https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/
    ##
    tolerations: []
    ## podManagementPolicy for the statefulset, allows parallel startup of pods
    ## ref: https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#pod-management-policies
    ##
    podManagementPolicy: OrderedReady
    ## updateStrategy for MongoDB Primary, Secondary and Arbitrer statefulsets
    ## ref: https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#update-strategies
    ##
    updateStrategy:
      type: RollingUpdate
    ## Entries for the MongoDB config file. For documentation of all options, see:
    ## ref: http://docs.mongodb.org/manual/reference/configuration-options/
    ##
    config:
    ## Name of a ConfigMap containing a MongoDB config file (cannot be used at the same time as config)
    ## ref: http://docs.mongodb.org/manual/reference/configuration-options/
    ##
    configCM:
    ## An array to add extra env vars
    ## For example:
    ## extraEnvVars:
    ##  - name: KIBANA_ELASTICSEARCH_URL
    ##    value: test
    ##
    extraEnvVars: []
    ## Name of a ConfigMap containing extra env vars
    ##
    extraEnvVarsCM:
    ## Name of a Secret containing extra env vars
    ##
    extraEnvVarsSecret:
    ## Add sidecars to the pod
    ## For example:
    ## sidecars:
    ##   - name: your-image-name
    ##     image: your-image
    ##     imagePullPolicy: Always
    ##     ports:
    ##       - name: portname
    ##         containerPort: 1234
    ##
    sidecars: []
    ## Add init containers to the pod
    ## For example:
    ## initcontainers:
    ##   - name: your-image-name
    ##     image: your-image
    ##     imagePullPolicy: Always
    ##
    initContainers: []
    ## Additional pod annotations
    ## ref: https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/
    ##
    podAnnotations: {}
    ## Additional pod labels
    ## ref: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
    ##
    podLabels: {}
    ## Array to add extra volumes
    ##
    extraVolumes: []
    ## Array to add extra mounts (normally used with extraVolumes)
    ##
    extraVolumeMounts: []
    ## Use an alternate scheduler, e.g. "stork".
    ## ref: https://kubernetes.io/docs/tasks/administer-cluster/configure-multiple-schedulers/
    ##
    schedulerName:
    ## Pod disruption budget
    ##
    pdb:
      enabled: false
      ## Use 0 to disable
      ##
      minAvailable: 0
      ## Use 0 to disable
      ##
      maxUnavailable: 1
    ## K8s Service Account.
    ## ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/
    ##
    serviceAccount:
      ## Specifies whether a ServiceAccount should be created for shardsvr.
      ##
      create: false
      ## The name of the ServiceAccount to use.
      ## If not set and create is true, a name is generated using the XXX.fullname template
      ##
      # name:

  ## Properties for arbiter nodes
  ## ref: https://docs.mongodb.com/manual/tutorial/add-replica-set-arbiter/
  ##
  arbiter:
    ## Number of arbiter nodes
    ##
    replicas: 1
    ## Configure resource requests and limits
    ## ref: http://kubernetes.io/docs/user-guide/compute-resources/
    ##
    resources: {}
    ## MongoDB additional command line flags
    ##
    ## Can be used to specify command line flags, for example:
    ##
    ## mongodbExtraFlags:
    ##  - "--wiredTigerCacheSizeGB=2"
    ##
    mongodbExtraFlags: []
    ## Pod priority
    ## https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/
    ##
    priorityClassName:
    ## Node selector
    ## ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselector
    ##
    nodeSelector: {}
    ## Affinity
    ## ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity
    ##
    affinity: {}
    ## Tolerations
    ## ref: https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/
    ##
    tolerations: []
    ## podManagementPolicy for the statefulset, allows parallel startup of pods
    ## ref: https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#pod-management-policies
    ##
    podManagementPolicy: OrderedReady
    ## updateStrategy for MongoDB Primary, Secondary and Arbitrer statefulsets
    ## ref: https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#update-strategies
    ##
    updateStrategy:
      type: RollingUpdate
    ## Entries for the MongoDB config file. For documentation of all options, see:
    ## ref: http://docs.mongodb.org/manual/reference/configuration-options/
    ##
    config:
    ## Name of a ConfigMap containing a MongoDB config file (cannot be used at the same time as config)
    ## ref: http://docs.mongodb.org/manual/reference/configuration-options/
    ##
    configCM:
    ## An array to add extra env vars
    ## For example:
    ## extraEnvVars:
    ##  - name: KIBANA_ELASTICSEARCH_URL
    ##    value: test
    ##
    extraEnvVars:
    ## Name of a ConfigMap containing extra env vars
    ##
    extraEnvVarsCM:
    ## Name of a Secret containing extra env vars
    ##
    extraEnvVarsSecret:
    ## Add sidecars to the pod
    ## For example:
    ## sidecars:
    ##   - name: your-image-name
    ##     image: your-image
    ##     imagePullPolicy: Always
    ##     ports:
    ##       - name: portname
    ##         containerPort: 1234
    ##
    sidecars: []
    ## Add init containers to the pod
    ## For example:
    ## initcontainers:
    ##   - name: your-image-name
    ##     image: your-image
    ##     imagePullPolicy: Always
    ##
    initContainers: []
    ## Additional pod annotations
    ## ref: https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/
    ##
    podAnnotations: {}
    ## Additional pod labels
    ## ref: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
    ##
    podLabels: {}
    ## Array to add extra volumes
    ##
    extraVolumes: []
    ## Array to add extra mounts (normally used with extraVolumes)
    ##
    extraVolumeMounts: []
    ## Use an alternate scheduler, e.g. "stork".
    ## ref: https://kubernetes.io/docs/tasks/administer-cluster/configure-multiple-schedulers/
    ##
    schedulerName:
    ## K8s Service Account.
    ## ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/
    ##
    serviceAccount:
      ## Specifies whether a ServiceAccount should be created for shardsvr arbiter nodes.
      ##
      create: false
      ## The name of the ServiceAccount to use.
      ## If not set and create is true, a name is generated using the XXX.fullname template
      ##
      # name:
  ## Enable persistence using Persistent Volume Claims
  ## ref: http://kubernetes.io/docs/user-guide/persistent-volumes/
  ##
  persistence:
    enabled: true
    ## The path the volume will be mounted at, useful when using different
    ## MongoDB images.
    ##
    mountPath: /bitnami/mongodb

    ## The subdirectory of the volume to mount to, useful in dev environments
    ## and one PV for multiple services.
    ##
    subPath: ""

    ## mongodb data Persistent Volume Storage Class
    ## If defined, storageClassName: <storageClass>
    ## If set to "-", storageClassName: "", which disables dynamic provisioning
    ## If undefined (the default) or set to null, no storageClassName spec is
    ##   set, choosing the default provisioner.  (gp2 on AWS, standard on
    ##   GKE, AWS & OpenStack)
    ##
    storageClass: "mongodb-sc"
    accessModes:
      - ReadWriteOnce
    ## PersistentVolumeClaim size
    ##
    size: 50Gi
    ## Additional volume annotations
    ## ref: https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/
    ##
    annotations: {}

## Config Server replica set properties
## ref: https://docs.mongodb.com/manual/core/sharded-cluster-config-servers/
##
configsvr:
  ## Number of replicas. A value of replicas=1 is simply a primary node
  ##
  replicas: 3
  ## Configure resource requests and limits
  ## ref: http://kubernetes.io/docs/user-guide/compute-resources/
  ##
  resources: {}
  ## MongoDB additional command line flags
  ##
  ## Can be used to specify command line flags, for example:
  ##
  ## mongodbExtraFlags:
  ##  - "--wiredTigerCacheSizeGB=2"
  ##
  mongodbExtraFlags: []
  ## Pod priority
  ## https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/
  ##
  priorityClassName:
  ## Node selector
  ## ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselector
  ##
  nodeSelector: {}
  ## Affinity
  ## ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity
  ##
  affinity: {}
  ## Tolerations
  ## ref: https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/
  ##
  tolerations: []
  ## podManagementPolicy for the statefulset, allows parallel startup of pods
  ## ref: https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#pod-management-policies
  ##
  podManagementPolicy: OrderedReady
  ## updateStrategy for MongoDB Primary, Secondary and Arbitrer statefulsets
  ## ref: https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#update-strategies
  ##
  updateStrategy:
    type: RollingUpdate
  ## Entries for the MongoDB config file. For documentation of all options, see:
  ## ref: http://docs.mongodb.org/manual/reference/configuration-options/
  ##
  config:
  ## Name of a ConfigMap containing a MongoDB config file (cannot be used at the same time as config)
  ## ref: http://docs.mongodb.org/manual/reference/configuration-options/
  ##
  configCM:
  ## An array to add extra env vars
  ## For example:
  ## extraEnvVars:
  ##  - name: KIBANA_ELASTICSEARCH_URL
  ##    value: test
  ##
  extraEnvVars:
  ## Name of a ConfigMap containing extra env vars
  ##
  extraEnvVarsCM:
  ## Name of a Secret containing extra env vars
  ##
  extraEnvVarsSecret:
  ## Add sidecars to the pod
  ## For example:
  ## sidecars:
  ##   - name: your-image-name
  ##     image: your-image
  ##     imagePullPolicy: Always
  ##     ports:
  ##       - name: portname
  ##         containerPort: 1234
  ##
  sidecars: []
  ## Add init containers to the pod
  ## For example:
  ## initcontainers:
  ##   - name: your-image-name
  ##     image: your-image
  ##     imagePullPolicy: Always
  ##
  initContainers: []
  ## Additional pod annotations
  ## ref: https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/
  ##
  podAnnotations: {}
  ## Additional pod labels
  ## ref: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
  ##
  podLabels: {}
  ## Array to add extra volumes
  ##
  extraVolumes: []
  ## Array to add extra mounts (normally used with extraVolumes)
  ##
  extraVolumeMounts: []
  ## Use an alternate scheduler, e.g. "stork".
  ## ref: https://kubernetes.io/docs/tasks/administer-cluster/configure-multiple-schedulers/
  ##
  schedulerName:
  ## Pod disruption budget
  ##
  pdb:
    enabled: false
    ## Use 0 to disable
    ##
    minAvailable: 0
    ## Use 0 to disable
    ##
    maxUnavailable: 1
  ## Enable persistence using Persistent Volume Claims
  ## ref: http://kubernetes.io/docs/user-guide/persistent-volumes/
  ##
  persistence:
    enabled: true
    ## The path the volume will be mounted at, useful when using different
    ## MongoDB images.
    ##
    mountPath: /bitnami/mongodb

    ## The subdirectory of the volume to mount to, useful in dev environments
    ## and one PV for multiple services.
    ##
    subPath: ""

    ## mongodb data Persistent Volume Storage Class
    ## If defined, storageClassName: <storageClass>
    ## If set to "-", storageClassName: "", which disables dynamic provisioning
    ## If undefined (the default) or set to null, no storageClassName spec is
    ##   set, choosing the default provisioner.  (gp2 on AWS, standard on
    ##   GKE, AWS & OpenStack)
    ##
    # storageClass: "-"
    accessModes:
      - ReadWriteOnce
    ## PersistentVolumeClaim size
    ##
    size: 8Gi
    ## Additional volume annotations
    ## ref: https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/
    ##
    annotations: {}
  ## K8s Service Account.
  ## ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/
  ##
  serviceAccount:
    ## Specifies whether a ServiceAccount should be created for configsvr.
    ##
    create: false
    ## The name of the ServiceAccount to use.
    ## If not set and create is true, a name is generated using the XXX.fullname template
    ##
    # name:

  ## Use a external config server instead of deploying one
  ##
  external:
    ## Host name of the config server primary node
    ##
    host:
    ## Root password of the config server primary node
    ##
    rootPassword:
    ## Name of the replica set
    ##
    replicasetName:
    ## Replica set key
    ##
    replicasetKey:

## Mongos properties
## ref: https://docs.mongodb.com/manual/reference/program/mongos/#bin.mongos
##
mongos:
  ## Number of replicas
  ##
  replicas: 3
  ## Configure resource requests and limits
  ## ref: http://kubernetes.io/docs/user-guide/compute-resources/
  ##
  resources: {}
  ## MongoDB additional command line flags
  ##
  ## Can be used to specify command line flags, for example:
  ##
  ## mongodbExtraFlags:
  ##  - "--wiredTigerCacheSizeGB=2"
  ##
  mongodbExtraFlags: []
  ## Pod priority
  ## https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/
  ##
  priorityClassName:
  ## Node selector
  ## ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselector
  ##
  nodeSelector: {}
  ## Affinity
  ## ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity
  ##
  affinity: {}
  ## Tolerations
  ## ref: https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/
  ##
  tolerations: []
  ## updateStrategy for MongoDB Primary, Secondary and Arbitrer statefulsets
  ## ref: https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#update-strategies
  ##
  updateStrategy:
    type: RollingUpdate
  ## Entries for the MongoDB config file. For documentation of all options, see:
  ## ref: http://docs.mongodb.org/manual/reference/configuration-options/
  ##
  config:
  ## Name of a ConfigMap containing a MongoDB config file (cannot be used at the same time as config)
  ## ref: http://docs.mongodb.org/manual/reference/configuration-options/
  ##
  configCM:
  ## An array to add extra env vars
  ## For example:
  ## extraEnvVars:
  ##  - name: KIBANA_ELASTICSEARCH_URL
  ##    value: test
  ##
  extraEnvVars:
  ## Name of a ConfigMap containing extra env vars
  ##
  extraEnvVarsCM:
  ## Name of a Secret containing extra env vars
  ##
  extraEnvVarsSecret:
  ## Add sidecars to the pod
  ## For example:
  ## sidecars:
  ##   - name: your-image-name
  ##     image: your-image
  ##     imagePullPolicy: Always
  ##     ports:
  ##       - name: portname
  ##         containerPort: 1234
  ##
  sidecars: []
  ## Add init containers to the pod
  ## For example:
  ## initcontainers:
  ##   - name: your-image-name
  ##     image: your-image
  ##     imagePullPolicy: Always
  ##
  initContainers: []
  ## Additional pod annotations
  ## ref: https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/
  ##
  podAnnotations: {}
  ## Additional pod labels
  ## ref: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
  ##
  podLabels: {}
  ## Array to add extra volumes
  ##
  extraVolumes: []
  ## Array to add extra mounts (normally used with extraVolumes)
  ##
  extraVolumeMounts: []
  ## Use an alternate scheduler, e.g. "stork".
  ## ref: https://kubernetes.io/docs/tasks/administer-cluster/configure-multiple-schedulers/
  ##
  schedulerName:
  ## Pod disruption budget
  ##
  pdb:
    enabled: false
    ## Use 0 to disable
    ##
    minAvailable: 0
    ## Use 0 to disable
    ##
    maxUnavailable: 1
  ## K8s Service Account.
  ## ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/
  ##
  serviceAccount:
    ## Specifies whether a ServiceAccount should be created for mongos.
    ##
    create: false
    ## The name of the ServiceAccount to use.
    ## If not set and create is true, a name is generated using the XXX.fullname template
    ##
    # name:

## Properties for all of the pods in the cluster (shards, config servers and mongos)
##
common:
  ## Use hostnames instead of IP addresses
  ##
  useHostnames: true
  ## Whether enable/disable IPv6 on MongoDB
  ## ref: https://github.com/bitnami/bitnami-docker-mongodb/blob/master/README.md#enabling/disabling-ipv6
  ##
  mongodbEnableIPv6: false
  ## Whether enable/disable DirectoryPerDB on MongoDB
  ## ref: https://github.com/bitnami/bitnami-docker-mongodb/blob/master/README.md#enabling/disabling-directoryperdb
  ##
  mongodbDirectoryPerDB: false
  ## System log verbosity level
  ## ref: https://docs.mongodb.com/manual/reference/program/mongo/#cmdoption-mongo-ipv6
  ##
  mongodbSystemLogVerbosity: 0
  ## MongoDB System Log configuration
  ## ref: https://github.com/bitnami/bitnami-docker-mongodb#configuring-system-log-verbosity-level
  ##
  mongodbDisableSystemLog: false
  ## Maximum peer node waiting timeout (in seconds)
  ##
  mongodbMaxWaitTimeout: 120
  ## Configmap with init scripts to execute
  ##
  initScriptsCM:
  ## Secret with init scripts to execute (for sensitive data)
  ##
  initScriptsSecret:
  ## An array to add extra env vars
  ## For example:
  ## extraEnvVars:
  ##  - name: KIBANA_ELASTICSEARCH_URL
  ##    value: test
  ##
  extraEnvVars:
  ## Name of a ConfigMap containing extra env vars
  ##
  extraEnvVarsCM:
  ## Name of a Secret containing extra env vars
  ##
  extraEnvVarsSecret:
  ## Add sidecars to the pod
  ## For example:
  ## sidecars:
  ##   - name: your-image-name
  ##     image: your-image
  ##     imagePullPolicy: Always
  ##     ports:
  ##       - name: portname
  ##         containerPort: 1234
  ##
  sidecars: []
  ## Add init containers to the pod
  ## For example:
  ## initcontainers:
  ##   - name: your-image-name
  ##     image: your-image
  ##     imagePullPolicy: Always
  ##
  initContainers: []
  ## Additional pod annotations
  ## ref: https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/
  ##
  podAnnotations: {}
  ## Additional pod labels
  ## ref: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
  ##
  podLabels: {}
  ## Array to add extra volumes
  ##
  extraVolumes: []
  ## Array to add extra mounts (normally used with extraVolumes)
  ##
  extraVolumeMounts: []
  ## K8s Service Account.
  ## ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/
  ##
  serviceAccount:
    ## Specifies whether a ServiceAccount should be created for all Pods.
    ##
    create: false
    ## The name of the ServiceAccount to use.
    ## If not set and create is true, a name is generated using the XXX.fullname template
    ##
    # name:

## Init containers parameters:
## volumePermissions: Change the owner and group of the persistent volume mountpoint to runAsUser:fsGroup values from the securityContext section.
##
volumePermissions:
  enabled: false
  image:
    registry: docker.io
    repository: bitnami/minideb
    tag: buster
    pullPolicy: Always
    ## Optionally specify an array of imagePullSecrets.
    ## Secrets must be manually created in the namespace.
    ## ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
    ##
    # pullSecrets:
    #   - myRegistryKeySecretName
  resources: {}

## Pod Security Context
## ref: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/
##
securityContext:
  enabled: true
  fsGroup: 1001
  runAsUser: 1001
  runAsNonRoot: true

## Kubernetes Cluster Domain
## ref: https://kubernetes.io/docs/tasks/administer-cluster/dns-custom-nameservers/#introduction
##
clusterDomain: bis.staging

## Kubernetes service type
## ref: https://kubernetes.io/docs/concepts/services-networking/service/
##
service:
  ## Specify an explicit service name
  ##
  name:
  ## Additional service annotations (evaluate as a template)
  ## ref: https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/
  ##
  annotations: {}
  ## Service type
  ## ref: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types
  ##
  type: ClusterIP
  ## External traffic policy
  ## ref: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types
  ##
  externalTrafficPolicy: Cluster
  ## MongoDB Service port and Container Port
  ##
  port: 27017
  ## Set a fixed service ClusterIP addrress
  ## ref: https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.16/#servicespec-v1-core
  ##
  clusterIP:
  ## Specify the nodePort value for the LoadBalancer and NodePort service types.
  ## ref: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport
  ##
  nodePort:

  ## Specify the externalIP value ClusterIP service type.
  ## ref: https://kubernetes.io/docs/concepts/services-networking/service/#external-ips
  ##
  externalIPs: []

  ## Specify the loadBalancerIP value for LoadBalancer service types.
  ## ref: https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer
  ##
  loadBalancerIP:

  ## Specify the loadBalancerSourceRanges value for LoadBalancer service types.
  ## ref: https://kubernetes.io/docs/tasks/access-application-cluster/configure-cloud-provider-firewall/#restrict-access-for-loadbalancer-service
  ##
  loadBalancerSourceRanges: []
  ## Extra ports to expose (normally used with the `sidecar` value)
  ##
  extraPorts: []
  ## Specify the sessionAffinity setting for the service. Can be "None" or "ClientIP".
  ## If "ClientIP", consecutive client requests will be directed to the same mongos Pod
  ## ref: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies
  ##
  sessionAffinity: None

## Configure extra options for liveness and readiness probes
## This applies to all the MongoDB in the sharded cluster
## ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/#configure-probes)
##
livenessProbe:
  enabled: true
  initialDelaySeconds: 60
  periodSeconds: 10
  timeoutSeconds: 5
  failureThreshold: 6
  successThreshold: 1
readinessProbe:
  enabled: true
  initialDelaySeconds: 60
  periodSeconds: 10
  timeoutSeconds: 5
  failureThreshold: 6
  successThreshold: 1

## Prometheus Exporter / Metrics
##
metrics:
  enabled: true

  image:
    registry: docker.io
    repository: bitnami/mongodb-exporter
    tag: 0.11.2-debian-10-r44
    pullPolicy: Always
    ## Optionally specify an array of imagePullSecrets.
    ## Secrets must be manually created in the namespace.
    ## ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
    ##
    # pullSecrets:
    #   - myRegistryKeySecretName

  ## String with extra arguments to the metrics exporter
  ## ref: https://github.com/dcu/mongodb_exporter/blob/master/mongodb_exporter.go
  ##
  extraArgs: ""

  ## Metrics exporter resource requests and limits
  ## ref: http://kubernetes.io/docs/user-guide/compute-resources/
  ##
  # resources: {}

  ## Metrics exporter liveness and readiness probes
  ## ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/#configure-probes)
  ##
  livenessProbe:
    enabled: false
    initialDelaySeconds: 15
    periodSeconds: 5
    timeoutSeconds: 5
    failureThreshold: 3
    successThreshold: 1
  readinessProbe:
    enabled: false
    initialDelaySeconds: 5
    periodSeconds: 5
    timeoutSeconds: 1
    failureThreshold: 3
    successThreshold: 1

  ## Metrics exporter pod Annotation
  ##
  podAnnotations:
    prometheus.io/scrape: "true"
    prometheus.io/port: "9216"

  # Enable this if you're using https://github.com/coreos/prometheus-operator
  serviceMonitor:
    enabled: true
    namespace: monitoring
    # fallback to the prometheus default unless specified
    # interval: 10s
    ## Defaults to what's used if you follow CoreOS [Prometheus Install Instructions](https://github.com/bitnami/charts/tree/master/bitnami/prometheus-operator#tldr)
    ## [Prometheus Selector Label](https://github.com/bitnami/charts/tree/master/bitnami/prometheus-operator#prometheus-operator-1)
    ## [Kube Prometheus Selector Label](https://github.com/bitnami/charts/tree/master/bitnami/prometheus-operator#exporters)
    ##
    selector:
      prometheus: kube-prometheus

[maxisam]

I wonder if we need to enable operationProfiling in the config. @juan131 do you know the correct syntax to do it ?

I tried this but no luck.

    config: |-
      operationProfiling:
        mode: all
        slowOpThresholdMs: 200
        slowOpSampleRate: 1

[maxisam]

@CCJY How did you get metrics of shardsvr and configsvr in the end? thx

[CCJY]

Hello. @maxisam, @juan131, @PhanSon95

First, for metrics of configsvr, shardsvr, and mongos, I believe there is another way that might be servicemonitor to collect metrics of configsvr, shardsvr, and mongos, but my solution is a little bit simple. Actually, I want to get an effective way, but I don't have enough time... By the way, It would be working for metrics.

For metrics of configsvr, shardsvr, and mongos

Step 1

I don't use servicemonitor in value.yaml. If you have a service monitor, you can delete it.

Step 2

Create file "mongodb-metrics-scrape.yaml"

In my case, I have 2 shardsvr, 1 configsvr, and 1 mongos

- job_name: "mongodb-sharded-shardsvr"
  metrics_path: "/metrics"
  static_configs:
  - targets: ['mongodb-sharded-shard0-data-0.mongodb-sharded-headless.mongodb-system.svc.cluster.local:9216']
    labels:
      instance: shard0
  - targets: ['mongodb-sharded-shard1-data-0.mongodb-sharded-headless.mongodb-system.svc.cluster.local:9216']
    labels:
      instance: shard1
- job_name: "mongodb-sharded-mongos"
  metrics_path: "/metrics"
  static_configs:
  - targets: ['mongodb-sharded.mongodb-system.svc.cluster.local:9216']
    labels:
      instance: mongos
- job_name: "mongodb-sharded-configsvr"
  metrics_path: "/metrics"
  static_configs:
  - targets: ['mongodb-sharded-configsvr-0.mongodb-sharded-headless.mongodb-system.svc.cluster.local:9216']
    labels:
      instance: configsvr

Step 3

kubectl create secret generic additional-scrape-configs --from-file=mongodb-metrics-scrape.yaml --dry-run -oyaml >additional-scrape-configs.yaml

kubectl apply -f additional-scrape-configs.yaml

Step 4

kubectl edit promethus

Step 5 Finally, add an additional configuration in prometheus.yaml

spec:
  additionalScrapeConfigs:
    key: mongodb-metrics-scrape.yaml
    name: additional-scrape-configs

Enable OperationProfiling

I tried that, but that's not working. Thus, I used another way.

Step 1

kubectl run --namespace mongodb-system mongodb-sharded-client --rm --tty -i --restart='Never' --image docker.io/bitnami/mongodb-sharded:4.4.2-debian-10-r0 --command -- mongo admin --host mongodb-sharded.mongodb-system.svc.cluster.local

Step 2

db.auth('root', 'YOUR PASSWORD');
use YOUR DATABASE0
db.setProfilingLevel(2); or db.setProfilingLevel(1); 

If it doesn't have system.profile on your db

you should connect to another shardsvr, then try again.

[maxisam]

@CCJY Thank you! However, I still wonder if there is a way to use this chart directly to get everything work.

It seems like you don't use replicaset in your setup, so I guess you don't have any metrics related to replicaset?

[CCJY]

@maxisam "I still wonder if there is a way to use this chart directly to get everything work." I have no idea now.
"It seems like you don't use replicaset in your setup, so I guess you don't have any metrics related to replicaset?" It seems no problem. I believe I got every metric. Please let me know if I need to do something for metrics.

[juan131]

Hi everyone,

I created a PR to address this issue by using PodMonitor(s) to scrap metrics. Find detailed information in the above PR