Sauna 0.2.0

[rzats]

Contains

• An initial implementation of a RT pipeline for Sauna. This includes the Kinesis observer as well as HipChat/Slack/PagerDuty responders. For specs & schemas, see:

  • Commands for analysts
  • Amazon Kinesis Observer setup guide
  • HipChat Responder setup guide
  • HipChat Responder user guide
  • Slack Responder setup guide
  • Slack Responder user guide
  • PagerDuty Responder setup guide
  • PagerDuty Responder user guide

• Third-party contributions from #66, #68 and #69.

• Closes #47, #53, #70, #71 and #75.

Outstanding before merging

• [x] I couldn't include iglu-scala-client. Including an igluClient dependency in Dependencies.Libraries, then in the main build.sbt causes this error:

  [warn]  ::::::::::::::::::::::::::::::::::::::::::::::
  [warn]  ::          UNRESOLVED DEPENDENCIES         ::
  [warn]  ::::::::::::::::::::::::::::::::::::::::::::::
  [warn]  :: org.scalaz#scalaz-core_2.11;7.0.0: not found
  [warn]  ::::::::::::::::::::::::::::::::::::::::::::::

  This means that SelfDescribing data isn't validated against its schema for now.

• [x] The command processing functionality is definitely not HipChat-specific and should be extracted into a more general class, potentially a base command-processing resolver.

• [x] The Kinesis observer should match the existing iglu-central spec...

• [x] ...and should be covered by a test suite.

• [x] A fix/implementation for #75 should be included.

[alexanderdean]

Thanks @rzats - can you talk to @chuwy please about the Iglu Scala Client...

[alexanderdean]

Removing @jbeemster as a reviewer as he is on holiday

[alexanderdean]

@chuwy can you do the first pass review please?

[chuwy]

@alexanderdean yes, will do.

[chuwy]

I'm also sorry that you have to work with this Iglu/play-json combination as most of functionality you need was already implemented in iglu core, but for json4s and circe libraries. We never used play json before.

[alexanderdean]

I'm also sorry that you have to work with this Iglu/play-json combination as most of functionality you need was already implemented in iglu core, but for json4s and circe libraries. We never used play json before.

What's the reason for using Play JSON?

[chuwy]

@alexanderdean it was picked by intern who started to write sauna, no other reasons.

[rzats]

@alexanderdean @chuwy: this is ready for another round of reviews.

A few notes:

• I've attempted to resolve #75 in 71bb12f, but this was reverted for now since unexpected side effects were caused by the changes (see test results) and the approach I've used was quite awkward, using lots of additional parameterization as well as requiring responders to override a ClassTag field due to type erasure. Will make another attempt at solving this later.
• The experimental iglu-scala-client version resolves fine, but causes critical internal errors when actually used (see test results). The dependencies for that library need to be updated ASAP - the code should work fine when that's done. Fixed!

[alexanderdean]

Thanks - @chuwy can you do another pass?

[rzats]

@chuwy: I've separated the function into two for now. Still not an ideal solution (the same amount of match blocks and LOC is the same), but it's not as nested - and allows us to programmatically generate Commands for tests etc.

[chuwy]

Hey @rzats,

I just pushed my proposal for #75 in https://github.com/snowplow/sauna/commit/b3c434969a64189949fdf16bcf7391e8b66625f6 (I didn't run tests, but it should be trivial to modify them).

I need to admit that parametrizing all events in 0.1.0 was a slight over-engineering. Type parameters there don't add any typesafety and don't generalize anything. Designer of responder always knows what exactly observer events he can parse into responder events, so type parameter here don't make much sense, usual subtyping works fine. This all becomes even more important taking in account akka design, erasure, etc.

[rzats]

@alexanderdean @chuwy: I've now implemented all the features from the 0.2.0 milestone. This should be ready for a final review!

[alexanderdean]

Awesome - thanks @rzats! Requesting the first pass review from Anton...

[rzats]

@chuwy: fixed most of the issues you've mentioned (besides command documentation - still need some clarification from @alexanderdean on what some of the metadata actually means). Also GitHub doesn't let me reply to your last comment, but yes, all the new environment variables are on Travis.

[chuwy]

Well done, @rzats! I also made another pass on Kinesis Observer and it looks fine.

Two last things:

• could you please remove override modifiers everywhere where it is not necessary. override usually signals developer that default class' behavior is overridden, but in most our cases that's not true, as those superclasses have no default behavior.
• could you update copyright years in files you edited/created to 2016-2017

[rzats]

@chuwy: done!

[alexanderdean]

Closing, will open a PR with the new branch.