Migrate builds to Gradle

[cdeszaq]

As nice as Maven is, it's been a challenge to get it working the way we'd like and making it do more seems to be more painful than painless. So, to ease our pain (and hopefully get faster builds), we want to move to Grade.

The main reasons we're looking at switching are:

1. Speed.

   Gradle is really good at only recompiling what it needs to, and while Maven can (in theory) do some of this, it's not nearly as good at it. Gradle, in general, has more flexibility (or at least more easily accessible mechanisms) around speeding up builds and determining dependencies.

2. Flexibility and Control

   Gradle is much more easily flexible than Maven is. Not only are there a bunch of plugins out there for Gradle already, but Gradle also makes it very easy to write new plugins for richer flexibility, whereas Maven does not do the same. One of these big things that we're looking at for this is the ability to have multiple sets of tests for different categories of things. In particular, breaking out the Unit, Functional, and Integration tests as separate aspects.

3. General Scriptability

   Gradle makes it much easier to do related build scripting, partly due to the way it views the world, as well as due to it's Groovy roots and the ease with which external processes can be managed. In particular, some of the things we're looking at are:

   1. Spinning up an instance of Fili using an embedded Jetty very easily (ideally with just a command like gradlew run
   2. Spinning up a full suite of demo dependencies (Druid, Redis, etc) so that a sandbox instance can be running in under 5 minutes with minimal setup with just a simple command like gradlew demo
   3. Spinning up real dependencies during tests so that we can test against real Druid instances, and potentially multiple versions, rather than having to fake everything

4. Developer Friendliness

   Gradle comes with gradlew, which is a wrapper script that will download the dependencies needed for a project to build / compile (on both Windows and *Nix environments), so that developers don't need to worry about having the right version of Gradle or other build dependencies installed. Gradle can handle that for us, which makes it easier to get up and going for those looking to extend Fili.

5. Better Groovy Support

   Maven's support for compiling both Java and Groovy in the same project is limited, rather clunky, and not super well supported. Not only has this caused some problems, but it's also made it more challenging to use Groovy-oriented tooling, like Code Narc (which is like CheckStyle for Groovy). Gradle, on the other hand, has multi-language support (especially Groovy and Java) as a 1st-class feature.

[archolewa]

💯 💯 💯 💯 💯 💯

[dejan2609]

@cdeszaq would love to work on this. Am I allowed to share some ideas / send PR (if external contributions are allowed for this specific ticket) ?

[michael-mclawhorn]

Right now I'm looking for some Fili implementor to pilot using gradle on their fili-derived project to figure out how difficult or easy it is to adapt the build process to it. Our chief gradle evangelist from the Core Team(@cdeszaq) moved and took a new job and has been fairly idle since then.

[cdeszaq]

By all means feel free to take a stab at it / provide input. Our Gradle efforts are very nascent so anyone with experience (or desire to learn) is welcome.

As Mike said, I've been more checked out than originally planned, but I am watching from afar more than it may seem ;-)

I will say that one of the biggest hurdles will be to help the whole Fili community get up to speed with Gradle and be comfortable with it. So with that in mind, lots of comments and even links to documentation so that others can follow along, and to explain why a choice makes sense, would be a good idea.

On Aug 14, 2017, at 1:19 PM, michael-mclawhorn notifications@github.com wrote:

Right now I'm looking for some Fili implementor to pilot using gradle on their fili-derived project to figure out how difficult or easy it is to adapt the build process to it. Our chief gradle evangelist from the Core Team(@cdeszaq) moved and took a new job and has been fairly idle since then.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[dejan2609]

@michael-mclawhorn I'll try to provide non-biased Maven vs. Gradle comparison (followed by some examples / pitfalls / suggestions).

Maven (devil's) advocacy combined with just a few Gradle pitfalls

(main idea is to show that Maven is not all that bad or that Gradle is not a silver bullet solution):

• Maven's Inflexibility Is Its Best Feature
• http://www.wakaleo.com/blog/what-has-maven-ever-done-for-us
• Gradle doesn't separate declarative bits from code, so builds descend rapidly into complex mess. Maven is boring but much more reliable https://twitter.com/jodastephen/status/888364803416367104
• old dog can learn some new tricks indeed: see my PR here #461 (maven wrapper related)
• Gradle tends to incubate some features for a way too long (example: maven-publish plugin): -- Gradle 1.4 (released on Jan 28, 2013) https://docs.gradle.org/1.4/release-notes.html#the-new-‘maven-publish’-plugin -- Gradle 4.1 (released on Aug 07, 2017, i.e. 4.5 years after 1.4) still lists same feature as incubating: https://docs.gradle.org/4.1/userguide/publishing_maven.html
• Maven isn't going anywhere anytime soon and newcomers must inherit some baggage: (https://github.com/gradle/gradle/issues/1162) Support consumption of Maven BOM natively

Final verdict:

• Maven: -- strict and conservative (one could even say: hermetic) but -- stable and consistent: virtually all maven build files are easy readable -- as opposed to many claims: even complicated stuff can be performed via maven (with a little help of some maven plugins)
• Gradle: -- has momentum (release cycle says it all) -- more powerful and certainly faster but -- build files can fall into oblivion quite easily (and become not so simple to maintain)

Some rules (among others) for successful Gradle builds:

• extract build logic into a separate module: -- https://stackoverflow.com/questions/13874680/what-is-the-purpose-of-gradles-buildsrc-folder/13875350#13875350 -- https://docs.gradle.org/current/userguide/organizing_build_logic.html#sec:build_sources
• use java-library plugin (where convenient) in order to separate API from implementation: -- https://docs.gradle.org/current/userguide/java_library_plugin.html
• use as little 3rd party plugins as possible (notable exceptions are well known plugins, especially those created by Netflix: https://nebula-plugins.github.io)
• use good practice promoted by major players (see next comment examples / links)

[dejan2609]

Examples / usefull links:

• SonarQube ongoing transition from Maven to Gradle: -- https://github.com/SonarSource/sonarqube/pull/1730
• https://julien.ponge.org/blog/an-experiment-in-maven-to-gradle-migration -- https://github.com/eclipse/golo-lang/pull/300
• http://blog.stack.foundation/2017/05/31/10-tips-for-migrating-from-maven-to-gradle

Big projects that moved from Maven to Gradle:

• Hibernate: -- https://hibernate.atlassian.net/browse/HHH-5616 -- https://github.com/hibernate/hibernate-orm/search?p=2&q=HHH-5616&type=Commits&utf8=✓
• Spring framework: -- https://github.com/spring-projects/spring-framework/pull/22
• Ehcache 3: -- https://github.com/ehcache/ehcache3/pull/7

Projects that use Gradle since day one:

• https://github.com/junit-team/junit5
• https://github.com/elastic/elasticsearch
• https://github.com/ReactiveX/RxJava
• https://github.com/mockito/mockito (nice hint: https://github.com/mockito/mockito/pull/641)

Rather complicated Gradle build examples IMHO (just to be fair: build requirements are high for these projects given a fact that their stack of technologies is pretty mixed OR they produce multiple/parallel artifacts):

• https://github.com/apache/kafka
• https://github.com/real-logic/simple-binary-encoding

[dejan2609]

Stay tuned, pull request (first draft) is on the way.