Logging analysis and options

[jekram]

We need logging analysis and options:

We should do a functional compare of logging services. For both Server and Client Side logging.

[sabachanna]

Asalam Alaikum

Here are options for logging.

https://drive.google.com/a/cloudkibo.com/file/d/0B27UJdKoodKMblpDbTFyQUtDYms/view?usp=sharing

I found many libraries that were open source.

[jekram]

Saba

Do you even read what I asked for?

1. I asked for logging service
2. I asked for function compare and analysis

I did not asked to you to do google search and give a raw data.

I am not going to waste my time implementing open source on our servers.

Jawaid

[sabachanna]

Asalaam Alaikum,

I could not understand function compare, I have listed down the features of each other so that we can see which service offers which features. There are some which does not offer storage but there are some which support storage of logs in cloud.

[jekram]

I am still wating for an analysis of variuous logging services. This was requested two days ago when you informed that we are lossing logging. Can I have this reponded today. I want a full analysis comparing functions/features and a recommendation.

[sabachanna]

Here are some other options which provide cloud-based logging services.

1) Paper trail app (https://papertrailapp.com)

• It does aggregation of data wherever it comes from.
• We can search data of specific date and time. Search in ranges of date.
• We can get alerts for specific log messages whenever they are logged.
• We can see the trends and can archive old data forever
• It has instant log visibility. Loggly took around 15 minutes to show the data which has just been logged.
• It is highly integretable and can be integrated with existing tools
• Free plan has 100 MB / month

2) Log entries (https://logentries.com/)

• It is also real time and can scale easily
• The speciality is that it is compatible with container environments like docker
• It also sends the alerts for specified log messages when they occur
• We can have custom tags and context views
• Real time monitoring and search is supported
• Good analytics tools and dashboard to visualize data
• Good features for data security like TLS
• It is highly integratable
• supports both structured and unstructured data
• Allows monitoring from mobile app
• Free trial 5 gb per month, 30 days free

3) Splunk light

• It generates real time log data
• It also sends alerts and notifications
• We can do real time reporting and analysis
• It also has dynamic dashboards
• It collects and indexes the data for us
• The dashboards are highly dynamic and can be zoomed in or out
• we can perform searches and calculate matrics
• alerts can be trigger

4) Loggly

• It is real time logging but take 15 minutes to load fresh data
• Search is data guided
• We can have bird eye view of data
• it has sophisticated guided search and analytics
• dynamic field explorer for logs
• full text search, search by fieldand boolean and by ranges
• we can have the trends and rich views
• we can also have real time alerts
• good dashboard and visualizations
• hihgly integrated and can integrate with dev ops

In respect of features they are not much different with few exceptions like one of them was suitable for docker environment too.

I would go in this priority of order

1) paper trial 2) log entries 3) loggly

[jekram]

Saba ASAK:

Let's do Papertrail

I have signed up.

Please use cloudkibo@cloudkibo.com as email address and cloudkibo as password.

You need to log in and setup the rest.

Jawaid

[sabachanna]

Ok Sir. I have included this in deck as task for today

[sabachanna]

Papertrail is now implemented. I have used the winston-papertrail library for nodejs

I took help from sojharo too as I was finding it difficult to integrate this library of papertrail.

We can see the logs on dashboard now

[sabachanna]

This is faster as compared to loggly, logs appear here instantly. In loggly, it took 10-15 minutes to show the recent logs

[jekram]

Saba ASAK:

Thanks.

We should use this in every meeting and in every testing.

Have we implemented it for both client and server?

Thanks

Jawaid

[sabachanna]

Asalam Alaikum

Yes Sir, implemented on client and server

[jekram]

Is this work done or more work is required?

[sabachanna]

This is done now.

[jekram]

Thanks