buep
commented
8 years ago ELK learnings - if someone want to try it...
- rivers are deprecated, instead there are input plugins, output plugins and filter and codec plugins. Last to datawashes.
- elastic search need for example logstash to get data, so logstash pushes data to elastic search. Use logstash plugin to get data: https://www.elastic.co/guide/en/logstash/current/index.html
Configure Jenkins server:
- when copying the URL, make sure to remove port and add that part to separate field
- I log to /logstash/jenkins as key
This job have been configured for post-build action, pushing the logs: http://code.praqma.net/ci/job/experimental-logstash-pretested-integration-plugin_verify-from-Github-Praqma_master/
It can also push each and every log console line, one by one. Checkbox above build steps.
Installed this plugin: https://wiki.jenkins-ci.org/display/JENKINS/Logstash+Plugin
Notes on getting the elastic cloud instance up and running
Open https://www.elastic.co/cloud, chose free trial and within minutes get access with your email.
In the Elastic cloud instance one get access to, create a minimal cluster. Enable Shield, and then enable Kibana. Then create a user under Security/Shield, but clicking "New user" and copy paste the password from plain-text in the first edit box. As soon as you click save, it is encrypted. Change the username, and save the password.
jenkins:6xll6fewadfafrfvol
In the second box, add the user to a role, eg. the admin role: admin: admin, jenkins Then the cluster is good, and you can log onto to Kibana, using the jenkins-user.
Jenkins and other build systems generate a lot of data that should be persisted. Not so much the build results, which is handled as artifact management system, but rather all the extra data that are worth trending.
So assume we store artifact somewhere, and we also store all relevant traceability information, then I still would like to keep data for trending and data analysis like number of failed builds, number of tests, failing tests, skipped test etc. For specifically Jenkins it could be any number of line in the console that would be analysed and converted to number that could be trended or analysed.
An ELK is one way to do it and I successfully tried that on a demo ELK stack in elastic cloud that isn't present anymore. See these used references:
Running it locally next time?
To save some work next time, I will add some learnings as a comment below.
Alternatives to an ELK stack could be Humio, or Datadog or Prometheus + Graphana based on some internal discussions we have.
This issue is a briefing and I propose we spend it on deciding on some user-stories we see as important related to persisting scratch data from our build systems.