Create monitoring environment

[Coenego]

Monitoring

In order to visualise the application activity, we need a box that stores the logs from the app nodes and houses the monitoring environment.

Since the previous stable release of Kibana connects with ElasticSearch over the web, we opt to use the current (beta) version, which is run as a service. Therefore we don't need to expose the ElasticSearch instance over HTTP. This also means that we need to use the current (beta) version of ElasticSearch, required by Kibana.

Repository: https://github.com/CUL-DigitalServices/avocet-monitoring

Dependencies

• ElasticSearch 1.4.0.Beta1
• Kibana 4.0.0-BETA1.1
• Logstash 1.4.2
• Nginx 1.6.2-1~precise 0
• Oracle JDK 8

  Application node

• [x] Add log stream (WIP)
• [x] Configure avocet-puppet scripts (WIP)

  Monitoring node

• [x] Spin up Amazon AWS instance
  • [x] Set up and configure new VM
  • [x] Install and configure dependencies
  • [x] Install and configure Puppet
• [x] Puppetise environment (WIP)
  • [x] ElasticSearch
  • [x] Puppetise ElasticSearch
  • [x] Nginx
  • [x] Puppetise Nginx
  • [x] Configure Nginx server
  • [x] Kibana
  • [x] Puppetise Kibana
  • [x] Create startup script
  • [x] Create ElasticSearch queries for Kibana
  • [x] Logstash
  • [x] Puppetise Logstash
  • [x] Create startup script
  • [x] Create configuration file. (WIP)

    Testing

• [ ] Test environment for staging server

  Future todo's

• [ ] Create Deamon script (WIP)

[bertpareyn]