add ability to publish java based plugins

[jsvd]

Currently jarvis executes a set of commands in order to publish a plugin:

  TASKS = { "bundle install" => ALWAYS_RUN,
            "bundle exec rake vendor" => ALWAYS_RUN,
            "bundle exec rake publish_gem" => ALWAYS_RUN }.freeze

However, for a java based plugin there are different commands that need to executed.

So the publish command needs to:

1. detect if the plugin is ruby or java based a) If ruby => execute current set of tasks b) If java => execute new set of tasks

[jsvd]

@danhermann what would be the list of actions for java? for building and publish gem we can either use a local jruby or use one from a logstash distribution.

[danhermann]

There is a gradle task named gem for Java plugins that produces a gem file with all of the Java plugin's code after bootstrapping all of the necessary JRuby dependencies. Once the gem file has been produced, I presume the steps to publish it to rubygems.org remain the same.

As you point out, there needs to be a way to detect whether the plugin is a Ruby or Java plugin. I propose to do that by looking for a gemspec file in the root of the plugin's repo. All Ruby plugins contain a gemspec file whereas the gemspec file for Java plugins is auto-generated only at packaging time.

Assuming the above works for distinguishing between Ruby and Java plugins, the one other thing that Java plugins need is a logstash-core-x.y.z.jar containing the actual plugin API in the co.elastic.logstash.api package. At some point, we may be able to publish that to Maven, but right now, it needs to be built from the Logstash repo. It also needs to be built from a specific branch of the Logstash repo so that plugins can be built against any version of the plugin API. I propose that version be read from a PLUGIN_API_VERSION file in the root of the plugin's repo though I am certainly open to other ideas for that. If that file is not present, it would default to 7.2, the Logstash branch containing the GA/1.0 version of the Java plugin API.

Assuming all of the above works, I think the following commands are all that would be necessary to accommodate Java plugins:

# starting from a directory containing the git repo for the plugin
git clone https://github.com/elastic/logstash/ --branch $PLUGIN_API_VERSION --single-branch ../logstash
cd ../logstash
./gradlew assemble # no need to bootstrap as only the logstash-core-x.y.z.jar is needed here
cd -
echo "LOGSTASH_CORE_PATH=../logstash/logstash-core" > gradle.properties
./gradlew gem
# once the gem is produced in the line above, the remaining steps to publish should be the same

[jsvd]

this is now in a feature branch being used by Jarvis https://github.com/elastic/jarvis/tree/java_plugins_publish once we have enough confidence we can merge it to master and have Jarvis use master again.