MongoDb: Adding availability to call function for collecting user specific metrics

[mishutka-mmx]

Feature Request

Opening a feature request kicks off a discussion.

Proposal:

For collecting user specific metrics it's very usfull call user defined function in MondoDb

Current behavior:

For Now, MongoDb plugin can not call user function;

Desired behavior:

For example, like in redis plugin realized it:

## Optional. Specify redis commands to retrieve values
  # [[inputs.redis.commands]]
  #   # The command to run where each argument is a separate element
  #   command = ["get", "sample-key"]
  #   # The field to store the result in
  #   field = "sample-key-value"
  #   # The type of the result
  #   # Can be "string", "integer", or "float"
  #   type = "string"

Use case:

1. To get different metrics (which not realized in base mongodb plugin ) based on rs.status()
2. To get business metrics based on user's collection data.

[powersj]

Hi,

Thanks for the feature request and your other issue reports around the MongoDB plugin.

With the ability to run random MongoDB functions, you may also get a wide variety of data types, like an array or nested objects. Did you have thoughts on how a user would parse that? Give the ability to run arbitrary commands, there could be a lot of random data that is returned.

Thanks

[mishutka-mmx]

Hello, We are using telegraf with Victoria Metrics. Victoria support only digital types as return metric value ( int\double\float)

According to my imagination:

1. We create some function in MongoDB For example

function myMetric(value){
db = db.getSiblingDB("mishadb");
collection = db.getCollection("tickets");
return collection.count({"amount": {$gt: value}}); 
}

MYRS:PRIMARY> myMetric(50)
12
MYRS:PRIMARY> myMetric(30)
16

2. Allow our monitoring user for executing function;
3. In telegraf's config adding something like this:

[[inputs.mongo.user_commands]]
#user_command = ["function()"]
#function - name of calling function , which exists in connected database
user_command = ["myMetric(50)"]
#Tag which is added to metric user_commands for function
tag_command  ="MyMetr"
# Type Can be "integer", or "float"
type = "integer"

I'm sure I'm wrong on the details somewhere. But It just concept of Plugin extension.

[powersj]

My concern is that if we allow users to run arbitrary commands, we have to expect arbitrary output. For example, you are showing someone running a custom function that returns one data point. Someone could also run and want to parse a value out of hello(), right?

[mishutka-mmx]

Hello, You are right.

We must declare some rules for user function.

Looking , like postgresql_extensible and redis done it

we can assume the following:

1. Function or collection query always must return projection like {"metric_name" : value } metric name - it is word without space value - metic value ( int\float)

So , pluging expected always

MYRS:PRIMARY> db.tickets.aggregate([{$match: {amount: {$gt: 50}}},{ $count: "amount" }])
{ "amount" : 12 }

Or multiple value

{ "amount1" : 13 }
{ "amount2" : 20 }

And based on

  #   # The type of the result
  #   # Can be "integer", or "float"
  #   type = "string"
  type = "int"

we could generate metrics like

mongodb_user_comand,tags1,tags2,.. amount=12i 1655358660000000000 mongodb_user_comand,tags1,tags2,.. amount1=13i 1655358660000000000 mongodb_user_comand,tags1,tags2,.. amount2=20i 1655358660000000000

or mongodb_user_comand,tags1,tags2,.. amount=12i,amount1=13i,amount2=20i 1655358660000000000

where tags 1,tag 2 - it is current generated tags by plugin

[powersj]

Function or collection query always must return projection like {"metric_name" : value }

Agreed, so for next steps: look at adding a custom query function that is similar to redis/postgresql_extensible where a user can create a custom query with the expectations that the returned value is of the format {"metric_name": value}.