jooby future

[jknack]

jooby 2

After a long consideration of existing implementation, ideas described in #926, evaluation of new frameworks (specially from Go but also http4k)... found some time to draw what and how Jooby 2 is going to look and work.

As always, your feedback is welcome.

design

• Single Context object replaces Request and Response
• Route Handler is now a function (returns a value)
• Filter is now a function and they are not tied to a route pattern. They are group and composed using as functions
• Route chain/pipeline replaced by functional filter
• Run code in the IO Thread
• Router algorithm now matches a single Handler. We can't add multiple handlers to same route
• No more greedy Ant path pattern: **
• There will be a clear difference between query, path and form parameters. Must get rid of the generic Request.param(String) method
• Root package will be io.jooby
• Remove DI/Guice from core

contracts

interface Context {
  //... merge of Request/Response objects from 1.x
}

interface Handler {
  Object apply(Context ctx);
}

interface Filter {
  Handler apply(Handler next);

  default Filter then(Filter next) {
    return h -> apply(next.apply(h));
  }

  default Handler then(Handler next) {
    return ctx -> apply(next).apply(ctx);
  }
}

interface Router {

  Handler match(String method, String path);

  //.. usual router methods
}

examples

HelloWorld

{
  get("/", ctx -> "Hello world!");
}

Nothing new, except we replaced the (Request, Response) signature by (Context)

Filter

{
  /** Timing filter: */
  filter(next -> {
    return ctx -> {
      long start = System.currentTimeMillis();
      Object response = next.apply(ctx);
      long end = System.currentTimeMillis();
      System.out.println("Took: " + (end - start));
      return response;
    };
  });

  get("/", ctx -> "Hello world!");
}

Here the timing filter applies to all the route defined bellow it.

Filters are not tied to a path pattern anymore, the Route.Chain was replaced by a function composition.

Scoped Filter

{
  group(() -> {
    /** JWT auth filter: */
    filter(new JWTToken());

    get("/api/pets", ctx -> ...);
  });
}

Here the JWT filter applies to all the route defined below which are wrapped by group operator.

unit tests

As expected unit test will be easily to write due we use a functional approach:

@Test
public void easyTest() {
   Context ctx = mock();
   Router router = ...;
   Handler handler = router.match("/");
   Object response = handler.apply(ctx);
   assertEquals("Hello world!", response);
}

path pattern

We are going to remove the greedy Ant path pattern, so patterns like:

• '/foo/**'
• '/foo/**/bar'

won't work any more

The new router algorithm will be a port of go-chi

thread model

We are going to keep the worker thread as default execution mode, also going to allow to execute a route in the IO thread:

{
    mode(Mode.IO); // Default is WORKER

    worker(ForkJoinPool.commonPool()); // Set a default worker or Fallback to `server` worker executor

    get("/", ctx -> "from IO thread");

    dispatch(() -> {
      get("/default-worker", ctx -> "from worker thread"); // Run with default worker executor
    });

    dispatch(executor, () -> {
      get("/executor", ctx -> "from custom executor thread"); // Run with a custom executor
    });
}

Filters are going run properly (not like in 1.x where they are ignored #996)

Just a quick preview of what is coming in 2.x

As always, your feedback is welcome.

[DerDackel]

I have a possible technical requirement: Make the complete Jooby build pipeline Java 9+ friendly. Currently compiling Jooby itself on Java 9+ seems to work, but many of the tests break, mostly due to the use of Powermock 1.x, which is not compatible with Java 9.

As Java 8 is supposed to go out of support some time after the launch of Java 11 this fall, if your're going for a clean slate anyway, this might be the best opportunity.

Sidenote: I've played around with current Jooby on JDK9 and once I replaced Powermock with the a build from HEAD of the 2.0.0 beta branch, everything but 3 tests in the morphia module ran fine. I'm a little concerned with the development of Powermock though right now: The only official beta build of 2.0.0 was released in September 2017 and there hasn't been a commit to the 2.x branch for two months now :(

[jknack]

Yes, good point. We should consider using Mockito... but not sure what to do with PowerMock

[DerDackel]

Not sure yet. The main use of Powermock I've come across is mocking static final fields. I think JMockit can do this too to a degree (IIRC it prepares tests using a java agent) but it has a completely different approach to testing and mocking than Easymock/Mockito. Beyond that I know no other alternative except avoiding static final fields.

[yholkamp]

Remove DI/Guice from core

Could you elaborate on what you've got in mind? For me one of the unique selling points of Jooby is that it's a 'micro' framework that supports DI/Guice and lets you build out a more complex app using the MVC routes. It'd be sad to see the support for DI or MVC routes removed or reduced.

[jknack]

Hi @yholkamp

I'm not sure yet :S, options are:

• build core without Guice and keep Module.java more or less as it is today (guice will be provided via additional module and/or when present in the class path)

• build core without Guice and define a custom and minimal DI. Then from module you could set default implementation to be Guice (or any other DI implementor). This was suggested for another Jooby user and this is what Play does: https://www.playframework.com/documentation/2.6.x/JavaDependencyInjection#play-libraries

Ideally, want to have Guice as a module and remove the Guava dependency from core.

[DerDackel]

I'm wanting to inject (wiggles eyebrows) Dagger 2 as another possible alternative as it does seem to keep the external dependencies lean, but this would mean a switch from runtime to compile time DI.

I wonder if this decision could be deferred to the Jooby user deciding which DI module they want to include in their build.

[jkbbwr]

Don't get rid of full fat MVC we have now

[paulovictorv]

I'll +1 the comment from @jkbbwr . I'm not a huge fan of functional programming and adding more features that depends on a functional idiom would be a negative point for me.. That's one of the main reasons I migrated away from Play Framework (besides the nasty code mix between java and scala and a heavyweight build tool [looking at you SBT]).

MVC Routes provide a clean way to define routes. It's way easier to organize code and browse through a large codebase (I'm thinking over 25 Controller classes) when using MVC when you compare with the functional style route declaration.

[jknack]

I do like MVC routes too, but think is possible to write large codebase with the functional version too.

In my mind MVC will be added as module/extension, but like DI/Guice would like to remove it from core.

What do you mvc users think if we use JAX-RS annotations? This was the initial idea for MVC in 1.x but then thought people will ask/complain about missing features of JAX-RS implementation and got writing our own set of annotations (Path, GET, Body, etc...)

Java 8 end of life is Jan 2019 (next year). Java 11 will be the next LTS version... I'm planning to make 2.x Java 11+ and keep 1.x with Java 8.

[jkbbwr]

What if we were to write MVC routes as a compiler plugin. You write MVC routes that are compiled into the functional style.

[jknack]

well, that will be awesome! do you mean a MVC->source code compiler? or MVC->bytecode?

[jkbbwr]

Id do it to source not bytecode. Write normal annotated java / kotlin, have it transpile into functional routes.

[jknack]

Like it. One additional benefit is that there won't be much overhead and doing source code it is clear what is going on either on success or error.

[mustafamotiwala]

Why do the additional transpile on the MVC Routes? If MVC Routes can be enabled as a plugin, then what does it matter if the bytecode was generated from an imperative style source code or functional source code? Is there a significant difference in the generated bytecode? Like the MVC style as it helps enforce the Single Responsibility Principle when organizing source code. I'd also like you to consider the adage - If it ain't broken, don't fix it. The MVC style isn't broken. In the interest of modularity it makes sense to pull it out as a module but would be cautious of too much experimentation on it.

Like the idea of pulling the DI part out of core. Suggest have a module provided by the client, and then the client can start the DI subsystem using the DI library of their choice. Specifically something akin to the aforementioned Play Framework Modules. Couple that with Dagger2 and compile time DI, and you have a very performant result.

My $0.02.

[mustafamotiwala]

What do you mvc users think if we use JAX-RS annotations? This was the initial idea for MVC in 1.x but then thought people will ask/complain about missing features of JAX-RS implementation and got writing our own set of annotations (Path, GET, Body, etc...)

👍 - JAX-RS compatibility makes learning Jooby easier and reduces the API surface area that one needs to learn.

[jknack]

@mustafamotiwala thanks for your input. Nothing is broken in Jooby 1.x 😄 just want to make it better 🚀

For example I built 1.x saying will never never run code in IO thread and/or use reactive libraries to build a web app....this was changing and end up adapting the existing code to support some of these requirements. In 2.x all these will have better support and implementation.

Mvc routes (as you pointed out) works OK, don't have much to complain about them. The only major requirement will be to removed it from core.

The existing implementation uses reflection to translate the core (functional route) into a MVC method. Here is where I think a source generator or byte code generator strategy will make them perform better (similar to functional routes) due we eliminate reflection

[jkbbwr]

I think source generator will be better than bytecode due to the fact it makes debugging easier, and will allow source mapping between the two. On top of that its more forward and backward compatible and simpler to reason about.

[nedtwigg]

I maintain a web application that started on Play Framework 1 back in 2009. When Play Framework 2 came out, we were never able to save up enough time to port, as it basically required a rewrite. The manner of the Play Framework 2 rollout made it difficult for users of the Play Framework 1 to maintain their applications - when you googled, you never knew if you were getting hits for Play 1 or Play 2. This was bad not just for existing Play 1 users, but also new users trying to adopt Play 2. Furthermore, as Play 2 churned through minor version bumps, it became clear that even if we had updated, it would have required significant effort to keep up with the platform changes.

Last year we finally got desperate enough to make the plunge to migrate our stack off of Play 1 onto something more modern. We knew from day 1 that we needed to pick a microframework so that we could update libraries at our own pace - libraries like netty live a lot longer than web frameworks, and we don't get much value out of feature-rich frameworks anyway for our application.

The changes you are suggesting for jooby 2 sound good, but I can guarantee you that our team will continue using jooby 1.x for years. Hopefully a whole entire decade longer. On our task list, "migrate from Jooby 1 to Jooby 2" is never going to become the highest-priority item, because none of our problems are due to limitations in jooby 1. It will always be easier for us to upgrade jooby 1's dependencies - and that's a testament to the great work of jooby 1.

All that said, I think it's great to reach for even higher heights with jooby 2, and I also think it's totally understandable for you to say "I'm working on Jooby 2 now, and I'd like to sunset my effort on Jooby 1". But I really hope you'll let the community continue to maintain Jooby 1 indefinitely, rather than to say that it is deprecated. Jooby 1 is so simple, I think it is possible to say that it is never deprecated, and continuously updated by the community of people who have built things on top of it.

There are only two hard things in Computer Science: cache invalidation and naming things.

From my experience with Play Framework, if the new jooby is called "jooby 2", and it is hosted at the same URLs and the same git repos as "jooby 1", I can guarantee that users of both jooby 1 and jooby 2 will have a bad experience.

One way to carry the jooby brand and community over to jooby 2, while still allowing jooby 1 to live forever is to call it "joobytwo", host it at github.com/jooby-project/joobytwo and joobytwo.org. This allows you a few things:

• googling for jooby and joobytwo will always be consistent
• URLs in blog posts about jooby 1 to https://github.com/jooby-project/jooby/ and https://jooby.org/ won't break
• because these URLs all continue to work and be valuable, you can use them to point people to jooby 2, with a message like Jooby will be forever maintained by the community that uses it, but there's a new library called joobytwo which is even better. We recommend starting new projects with joobytwo instead of jooby, but jooby will continue to get bugfix releases forever.
• if you want, you can hand off responsibility for jooby 1 to the crusty lazy people who can't be bothered to update
• besides the benefit to users of jooby 1 who can't update, it also helps people to adopt joobytwo. It tells them "joobytwo is never going to end-of-life underneath you. If a big breaking change is ever needed, they'll make joobythree and you can keep using joobytwo, just like those other dinosaurs stuck on jooby"

Stability is a highly underrated feature for web platforms, and it can be achieved just with careful naming :) If you follow this approach, there are two extreme outcomes:

• A: after a year, 100% of your users are on joobytwo, and jooby 1 is a ghost town
  • outcome: jooby 1 requires no maintenance, and clearly redirects people to joobytwo
• B: after a year, 100% of your users are still using jooby 1
  • outcome: it turned out that joobytwo doesn't bring enough new value to justify the switching costs for your users. You were able to learn this lesson without hurting the jooby brand

I think the only downside of the joobytwo name is that there's not enough "pushing" people to upgrade. If you end up with outcome B, maybe it will feel like "I should have pushed them more to switch by calling it jooby 2 and end-of-lifing jooby 1". But I think what that would actually tell you is that your users valued the stability you gave them in jooby 1 more than the conceptual elegance of you gave them in joobytwo. As a programmer, I love conceptual elegance. But as a program manager, I crave stability even more. Why not give your community both?

[jknack]

Thank you @nedtwigg really good feedback.

My plan is to release Jooby 2 in a new domain: jooby.io (new website). Once released I'm planning to have a 2.x branch in this same repository and probably after a year switch and move 2.x as master and create a 1.x branch

I have more than 20 application running today in production and I'm not planning to migrate them to 2.x so I agreed that Jooby 1.x will be here for a long time but probably I'm not going to add new features (modules) and/or fix some bugs that require a complete redesign of some Jooby components. Example of these issues are the issue you reported #920 but also #825 or #1093

The reason of using the Jooby brand and same repository are:

• jooby is now a recognized alternative for building web application in Java
• github repository has now 850+ star. This may sound silly to you (and to others too) but after 4 year of Jooby this is all we got. I'm sure not all users star the project but this is something important to get/collect more audience. If I move Jooby 2 development to a new repository will be in the same dilema :(

Your Play example is a very good one, but also believe they did it wrong. For example Netty (the default web server) changes their API at least 3 times (since I started using) and the Netty 5 version is going to change it again.

Spring 5 webflux (or whatever they call it) is another example of new API design.

[jknack]

Oh and there is one more factor to keep in mind: Java releases.

Jooby 1.x runs OK in Java 9+, but build simply fails in 9+ due mainly to mocking libraries

Jooby 2 requires Java 11+ and should move as fast as Java releases.

Java world is changing and moving faster than ever

[nedtwigg]

Sounds good! To be fair, the change from Play 1 -> 2 was much bigger than Jooby 1 -> 2, so Jooby can probably get away with more common-branding than Play could. Really glad that Jooby 2 will have its own website so that old links to 1.0 stay working.

Totally understand if you don't want to add new modules yourself, but I hope you'll be open to PR's from the community. I guess I'll just say that whether you allow new modules or not, people will probably keep writing them, and it's just a matter of whether they stay siloed or if there's a community place they can be contributed back to. It's a lot of work to maintain that community, and you're under no obligation to do that work, but I hope you'll consider open open sourcing rather than freezing it if at some point you decide that you're spending too much time on Jooby 1.

Cheers!

[jknack]

but I hope you'll be open to PR's from the community

Absolutely!! and will happy to fix core/server bugs for 1.x. If we are lucky those bugs going to be minors. Jooby 1.x is solid enough (4 years being in prod for me) and have a rich set of modules for almost in kind of web application

[dagnelies]

Hi,

I'm just starting to know Jooby and this topic piked my interest.

First, a dumb question: is jooby "2" just jooby 1 with more features ...or does it break completely break compatibility? ...that would be a big turn off IMHO. I don't want to have to rewrite my app every couple of years. It would also split the community, the effort, etc. It would suck.

As for the wishlist of extra features, the one I would have would be:

• using JAX-RS annotations, even if you just support the basic ones ...it's so large by now that full compliance would be hardcore overkill. However, reusing the same "usual" annotations is more familiar and makes porting easier.
• auto discovery! Writing tons of use(...) is alright, but if modules would self register and "REST" classes being picked on the fly, it would be even nicer!

[jknack]

Jooby 2.x is going to break (as explained before) and that is why we are going to release a 2.x version. It is a major release and on major release we are free to break compatibility.

Don't want Jooby converts into what other popular web frameworks are today: bloated software just because they need to keep backward compatibility.

I warranty that every major version is going to be stable (of course). I do have lot of Jooby 1.x deployments in prod and I'm not planning to move (most) of them to 2.x. Jooby 1.x sticks to Java 8, but I want Jooby 2.x start getting more features from Java it self (JDK 12)

Finally the two feature you suggested both will break compatibility:

1. JAX-RS. This means remove existing MVC annotations. Worst, support both :(
2. Impossible to do it write now due on how routing algorithm works today. We need to introduce an order, precedence, etc...

[jknack]

Jooby 2.0.0 released https://jooby.io/