Jinkies

Jinkies is an application for monitoring remote CI servers and notifying you of important build events (i.e. breaks and fixes). Rather than send emails which may go unnoticed, Jinkies notifications are intended to be attention grabbing. This version of Jinkies only supports mp3 files. In future we hope to support other audio content as well as video, text to speech, lava lamps, firework displays and Mars landings.

One of the nice features of Jinkies is that it allows you to "theme" build jobs. In this way you can assign a theme (e.g. "The Simpsons" or "Scooby Doo" etc) to groups of related jobs, and play appropriate content from those themes when an event occurs.

Non CI related events can be posted to Jinkies over HTTP, so it's also possible to use it to notify you of infrastructure related events such as backup failures or downed servers.

Danger Will Robinson!!!

Installing Jinkies on a publicly accessible server is a VERY bad idea. Doing so would allow anyone to upload and execute programs on that machine.

Quick Start

1. Check the system requirements
2. Install Jinkies
3. Add one or more build jobs

System Requirements

You will need a recent version of Java (1.6 should do) and a CSS3 / HTML5 compliant browser. Jenkins (and possibly Hudson) are currently the only supported CI servers. You'll also need a computer capable of playing sound, on which to install Jinkies.

Installation

Option 1 - Executable War

1. Download the executable binaries

2. Run 'java -Dhost=<host> -Dport=<port> -jar jinkies.war'

3. Wait for the embedded application server to start (you will see a line like the following)

   Jinkies has started successfully and can be accessed on http://server:port/

4. Point your preferred web browser at the specified host & port

Option 2 - Deploying to an Application Server

1. Download the deployable binaries

2. Create an external configuration file and add the line

   grails.serverURL = 'http://&lt;host&gt;:&lt;port&gt;[/&lt;context&gt;]'

3. Deploy the war file to your app server

4. Wait for the embedded application server to start (you will see a line like the following)

   Jinkies has started successfully and can be accessed on http://server:port/context

5. Point your preferred web browser at the specified host, context and port

Troubleshooting

• The first time you start Jinkies, it will take a little time to initialise it's database, please be patient.

• The default java memory settings are quite stingy. If you get memory related exceptions try increasing the JVM memory via the -Xmx and -XX:MaxPermSize arguments.

Monitoring CI Jobs

You can add Jenkins jobs one by one, or add every job on a specific server. Either way the process is the same...

1. Open the jobs page (e.g. http://jinkies-host:8080/jobs)
2. Click "Add Jenkins jobs..."
3. Enter a url (e.g. build.acuminous.meh:8080 or build.acuminous.meh:8080/job/jinkies)
4. Optionally specify a theme
5. Click "OK"

Jinkies is configured with some default content (thanks to freesound.org), so you should start hearing notifications a few seconds after adding your first job.

Creating Themes & Uploading Theme Content

The Jinkies default content is pretty limited. We'd love to have shipped it with our Scooby Doo, Star Wars and Tron sound samples, but were too worried about copyright infringment. To compensate we've tried to make it easy to add your own...

1. Open the themes page (e.g. http://jinkies-host:8080/themes)
2. Click "Add content..."
3. Say where the content should be sourced from - your computer, the interwebs* or text input
4. Select the file, enter the URL or type in the text
5. Optionally add a description
6. Enter themes the content is associated with (e.g. 'Scooby Doo')
7. Tick "Success" or "Failure", or specify a custom event type.
8. Click OK

Now whenever a job with this theme raises a matching event, this content becomes eligible for use. The actual content used in the notification is selected by random from all eligible content.

If instead of creating your own themes, you would like to add more content to the default theme, use 'Fallback' for the theme name

* We like http://www.rosswalker.co.uk/movie_sounds/ although the wav files from this site need some love before they will work.

Content Types

MP3

Right now Jinkies can only play mp3s. We're investigating reliable ways to play other file formats (wav, ogg & flac), but when it comes to java development we're much more familiar with the boring server side stuff, so it might be some time. For now we recommend converting your files to mp3 using MediaIO who have a much better idea what they're doing.

Text To Speech

We've taught Jinkies to read. If you upload a plain text file or type in the text directly, Jinkies will treat it as audio content and read it aloud. You can even reference variables and include any other valid Groovy Template syntax. For example:

Build ${build.number} of project ${job.displayName} resulted in ${build.result}

The following variables are available when a success or failure event occur

• build.number
• build.project
• build.result

If an error occurs while checking a remote build server then

• job.displayName
• type

Or for custom events

• target
• type

When previewing the text-to-speech function, the variables won't exist so Jinkies substitutes "var 1", "var 2", etc.

Other

Jinkies doesn't currently support any non-audio content. We're keen to add video and IP9255 support, but haven't got there yet.

Proxy Configuration

You can configure Jinkies to use a proxy server for HTTP and HTTPS traffic by following the standard Java Networking and Proxies configuration instructions.

HTTP Authentication

If your CI server requires authentication, then we recommend creating a read-only "jinkies" user on the CI server for this purpose. Currently the only way to tell Jinkies to use a username & password is to embed them in the job URL, e.g. https://bob:secret@build\.yourcompany\.com

This is just about OK if your CI server uses HTTPS (because the URL will be encrypted), but weak if it's fronted by Apache running HTTPS (because your password will be in clear text between Apache and the app server), but totally insecure if you're not using HTTPS and could easily lead to your server being pwnd if it's publicly accessible. (hint: Don't make your CI server publicly accessible without HTTPS!).

If this is unacceptable in your environment, let us know and we'll consider an improved solution. If you can't wait then place to start is uk.co.acuminous.jinkies.HttpClientsFactory.groovy. We look forward to your pull request.

HTTPS/SSL Configuration

If you're attempting to download content or connect to a build server over HTTPS and the remote server is using a self signed certificate you'll need to tell the JVM running Jinkies to trust it by downloading the certificate and adding it to the cacerts file using the java keytool program.

External Configuration

Jinkies is a Grails application. As such it has a main configuration file called Config.groovy and some environment based overrides such as development.groovy and production.groovy.

Jinkies will also attempt apply overrides from an external config file if such a file exists. By default Jinkies will look for /etc/jinkies/config.groovy, but you can change this by adding a -Djinkies.config=/path/to/config-file at startup.

Further information about Grails configuration can be found here.

Changing The CI Server Poll Frequency

Jinkies is configured to poll all jobs every 15 seconds. To change this setup an external configuration file, paste in the contents of QuartzConfig (see below), and set the 'repeatInterval' to the desired number of milliseconds.

import grails.plugin.quartz2.ClosureJob
import org.apache.log4j.Logger
import org.quartz.impl.triggers.SimpleTriggerImpl

grails.plugin.quartz2.jobSetup.jenkinsMonitor = { quartzScheduler, ctx ->

    def job = ClosureJob.createJob([name: 'JenkinsMonitorJob', concurrentExectionDisallowed: true]) { jobCtx , appCtx->
        try {
            appCtx.jenkinsMonitor.check()
        } catch (Throwable t) {
            Logger.getLogger('JenkinsMonitorJob').error('An error while handling Jenkins build events', t)
        }
    }

    def trigger = new SimpleTriggerImpl(
        name: 'Jenkins Monitor Trigger',
        startTime: new Date(),
        repeatInterval: 15000,
        repeatCount: -1
    )

    quartzScheduler.scheduleJob(job, trigger)
}

Scheduling Notifications

We use Jinkies to tell everyone it's time for the daily stand-up. Someday we hope to build a nice UI to do this, but right now you need a bit of HTTP and a text editor. To scheduling a notification, first setup an external configuration file, then paste in the following...

import org.apache.log4j.Logger
import org.quartz.impl.triggers.CronTriggerImpl 
import grails.plugin.quartz2.ClosureJob
import uk.co.acuminous.jinkies.util.HttpClientsFactory

grails.plugin.quartz2.jobSetup.projectXStandup = { quartzScheduler, ctx ->

    def job = ClosureJob.createJob('ProjectXStandup', { jobCtx , appCtx->
        try {
            Map params = [sourceId: 'project/x', theme: 'Scooby Doo', event: 'Stand-Up', channel: ['audio']]
            new HttpClientsFactory().getHttpBuilder().post(uri: 'http://localhost:8080/api/event', body: params)
        } catch (Throwable t) {
            Logger.getLogger('ProjectXStandupJob').error('An error while sounding the Project X standup', t)
        }
    })

    def trigger = new CronTriggerImpl(
        name: 'ProjectX Stand-up Trigger',
        cronExpression: '0 30 9 ? * MON-FRI' // 09:30 Monday - Friday
    )

    quartzScheduler.scheduleJob(job, trigger)
}

Substitute 'projectX' with your project name, 'Scooby Doo' with your theme etc and give the trigger a valid cron expression that represents when your Stand-Up event will fire. You can find more information about cron expressions here

Error Handling

If Jinkies encounters an error when retrieving or processing a build event it attempts to notify you using the Job's channel and theme. This could get very annoying if your build server is temporarily offline so Jinkies is configured to suppress repeated errors for one hour. You can override this by creating an external configuration file and assigning a supression duration.

jinkies.events.suppressRepeatedErrors = 60 * 60 * 1000

When Jinkies recovers from an error (e.g. the build server came back online) it triggers an internal 'Recovery' event, however there is no default content this, so no notification will be given. We did this because recovery notifications
could easily be mistaken for successes, and while the build server is now available again, the a job's status might still be failure. You can still upload your own content and assign them to the 'Recovery' event if you so wish.

Custom Events

If you want to use Jinkies to report other events, you need to POST a request to /api/event with the following parameters...

There must be at least one piece of content for the given event and theme (or 'Fallback' theme if you didn't specify one).

Duplicate Checking

By default Jinkies purges events that are a day old (although it will always keep at least one event in order to detect state changes). This means that it will not detect duplicate events where the original was older than one day. You can override this by creating external configuration file and assigning a time to live.

jinkies.events.ttl = 24 * 60 * 60 * 1000

The housekeeping job that purges old events is run nightly at 03:00. To change this paste the following into the external configuration file and update the cron expression

import org.apache.log4j.Logger
import org.quartz.impl.triggers.CronTriggerImpl 
import grails.plugin.quartz2.ClosureJob
import uk.co.acuminous.jinkies.util.HttpClientsFactory

grails.plugin.quartz2.jobSetup.eventHouskeeper = { quartzScheduler, ctx ->

    def job = ClosureJob.createJob([name: 'EventHousekeeperJob', concurrentExectionDisallowed: true]) { jobCtx , appCtx->
        try {
             appCtx.eventHousekeeper.run()
        } catch (Throwable t) {
            Logger.getLogger('EventHousekeeperJob').error('An error while tidying up old events', t)
        }
    }

    def trigger = new CronTriggerImpl(
        name: 'Event Housekeeper Trigger',
        cronExpression: '0 29 9 * * ?' // 03:00 Daily
    )

    quartzScheduler.scheduleJob(job, trigger)
}

Developer Notes

STS complains about compilation errors in Spock tests that use the @Build annotation immediately after a clean. Making a superficial change to the test causes it to be rebuild and subsequently work. We're gradually phasing out the build-test-data plugin because of this and other problems.

And Finally...

We hope you enjoy using Jinkies. Please do provide feedback (especially the negative kind).