New index for application

[cgonzalezg]

Hi, first of all nice work. My question is if it is possible to create a new index in the ES for every application that I want to monitor. Because right now all the information that I log is going to the same ES and same index, and it is dificult in kibana to check which application I am visualising.

Thanks

[seabaylea]

Currently for every record entry we store the same "header" to make it possible to identify the application/process: "hostName": {"type": "string", "index": "not_analyzed"}, "pid": {"type": "integer"}, Admittedly that's not much to go on if you have several different applications that run on the same machine (as pid doesn't tell you which application it is).

I am intending to make the index used configurable, but adding an extra field in the config object. That would let you use a separate index for each application.

Do you think there's other values that should be added to the record "header" that would help to identify the application?

[cgonzalezg]

Thanks for the fast response.

for me only with a field that identify  the application should be enough, I would like to group all the applications metric's when I have the same application running in different server or containers,  so the "hostName": {"type": "string", "index": "not_analyzed”} is not enough when I want to scale my application. 

My suggestion will be to set the application name in the configuration 

var config = {
    hosts: [
        'https://es1.bluemix.net',
        'https://es2.bluemix.net'
    ],    
    application: 'app',
    ssl: {
        ca: fs.readFileSync('./cacert.pem'),
        rejectUnauthorized: true
    }
}

[cgonzalezg]

so the same solution that you gave ;)

[seabaylea]

There's two way's we could approach that:

1. Add a "name" or similar field to each record that contains the name of the application, which defaults to the name of the .js file used to start the application (eg. app.js), but that can be explicitly set via the config object
2. Use the index as the name of the application, defaulting to "appmetrics" as it is now.

Would one way vs. the other make it easier for your to group and query?

[cgonzalezg]

I will go with the second approach, to isolate the metrics from the different applications in several indexes, so we will be sure if we delete a index it will only affect to the metrics of one application (microservice approach I will call it;)), although the second approach will let us compare the metrics of different applications and could be really cool to compare application version’s to see how the new changes affect the performance.

Now I am not sure ;)

[seabaylea]

@cgonzalezg Sorry for the delay in this - I've been travelling for the last week. I've landed two changes - one to make the index configurable, and one to add a configurable "applicationName".

You can add the configuration as follows:

var config = {
    index: 'myindex',
    applicationName: 'appName'
}
var appmetrics = require('appmetrics-elk').monitor(config);

Can you give it a test and see if it covers what your looking for? If it works for you, I'll deliver an update via npm.

[seabaylea]

I'll take no comment to mean "it works perfectly" ;-)