CDK-546: Initial version of a Spark demo.

[tomwhite]

This example has lots of boilerplate that I don't think belongs in an example. I think it should focus on the task of running a Spark job, and not add a CLI (including scripts and all the Maven build to do this) as a part of the example. Also, we should use the Kite CLI (or Maven plugin) to create and delete datasets.

[tomwhite]

A README is needed too. Can an integration test be written for this example?

[joey]

Can an integration test be written for this example?

Maybe. This example requires CDH5, so we would need a way to have the integration test for this example only run on CDH5 clusters.

[rdblue]

To detect CDH5 and run tests accordingly, we can use some of the dynamic helpers and junit's Assume

[joey]

I filed CDK-586 to do the integration test in a follow-up issue.

[joey]

FYI, you need to apply CDK-479 to your checkout of the kite project and do a mvn install on your cluster or VM before you can run this example.

[tomwhite]

This still needs a README that explains how to run the example.

[joey]

Sorry, the README was checked in but not pushed. It should be there now.

[joey]

I updated the PR with all of the review feedback. I made some changes to CDK-479 to be able to simplify the POM here. I'll go ahead an merge CDK-479 since that previously had a +1. Let me know if anything else is required on this issue.

[tomwhite]

+1

[joey]

Merged in cbc2a0a