Add a pre-build step in order to mark certain core Vavr methods as @GwtIncompatible

[danieldietrich]

Hi,

with modularization, I want to get rid of the @GwtIncompatible annotations of the core Vavr lib.

Goal/Acceptance Criteria:

Let the GWT compiler know which methods are not compatible without using the @GwtIncompatible annotation(s) within the core Vavr library. For example this could be achieved by adding a pre-build step to the vavr-gwt Maven build.

See this Twitter conversation about how to use [ByteBuddy]() to solve this issue.

In detail, I would suggest to explode the dependencies of vavr-gwt into the file system in a pre-build step. Furthermore we are able to rewrite/enhance the classes of our dependencies by dynamically annotating the according methods with vavr-gwt's own @GwtIncompatible annotation. After that, the GWT compiler is expected to work as desired.

I will add the list of methods later here.

[niloc132]

One word of caution: it looks like ByteBuddy modifies the bytecode of the classes rather than generate new sources, and GWT acts solely on source code, so will not see those new annotations if added only to bytecode without also modifying those original sources.

[danieldietrich]

@niloc132 thank you! It should work the same way. We need a pre-build step that augments the source(-jars) with the annotations (given a list of methods).

This is even easier because the GWT server-side (aka non-javascript code) can depend on the original Vavr jars.

We only need a tool that transforms the Java source. I‘m willing to write that but I would expect that the GWT eco system already has a solution for that purpose?

It is clean to separate the shortcomings of a third party library (like GWT) from our Vavr core library.

[niloc132]

I'm not aware of any such tool - other libraries like Google's Guava have the @GwtIncompatible annotation in the code where appropriate. Such a tool might want to be careful about mangling line/column numbers by adding those annotations, so that compile errors or runtime stack traces are still comparable across the same version of the tool.

In cases where it is too intrusive to add annotations (especially where inheritance needs to change, or new members need to be added to support compiling to JS), often the "super source" mechanism is used, where the GWT compile can be given a different copy of sources, with unusable members removed, etc, but I likewise haven't seen anyone doing that work in an automated way either.

[nfekete]

I'm not sure about the problem the removal of the @GwtIncompatible annotation was trying to solve. Can you elaborate a bit about that, @danieldietrich?

[danieldietrich]

@nfekete it is about modularization, more specifically module boundaries.

Types that are required by 3rd party tooling have no place in Vavr's core modules (like vavr-core, vavr-collection, vavr-control, ...). Even if only needed at compile time. I want to keep the code-base clean.

What if next there come libraries Abc and Xyz, which also require additional annotations? The rule is simple - we do not add any non-native annotations that have no other purpose than enriching static semantics of the source code (in a non-standard way).

It is a bad design of GWT to require other libraries to add GWT specific information. I don't understand why GWT only relies on annotations for source-code processing. For example the GWT compiler could be fed with xml's that contain method exclude lists. There are many other solutions to solve it.

One other solution is to add a pre-compile step that

• explodes source jars (maybe flattens them)
• modifies source files based on a list of GwtIncompatible code parts

I think such a tool could be interesting for the GWT community.

[danieldietrich]

We need something similar as ByteBuddy, for source code - a 'SourceBuddy' that transpiles resp. transforms Java sources based on the AST and a specific rule set.

But I think a fancy regexp / string replacer will also do the job. In JavaScript/TypeScript such string manipulations are relatively simple because regexps are baked into the language.

[ibaca]

GWT transpiles the Java source code to javascript. So this is not a third party tool, this is the whole vavr source code JRE and GWT compatible. But yep, assuming the GWT compatible responsibility might be something that vavr don't want to assume.

I am already doing something similar to your suggestion with RxJava (https://github.com/ibaca/RxJava/blob/2.x-gwt/build.gradle#L258) but it is not a great solution and requires manual (dangerous) work. I modify manually the source code and rebase the changes on each release. Then use a script to detect the changed sources and copy to the gwt adapter library. I don't think this is a GWT design problem, GWT is a transpiler, if you want your source code to be transpilable you need some minimal adaptation in it. The problem here is that WE want YOUR source code to be compatible 😉. RxJava chooses not to assume any change 👎.

It will be really appreciated if vavr tolerates a minimal source code adaptation to make it GWT/J2CL compatible (so transpilable to JS). But this is not a thirdparty lib, this is vavr assuming some minimal code adaptation to be compilable, which is awesome and very appreciated by the GWT/J2CL community 🙌. Thanks! super nice library btw!

[danieldietrich]

@ibaca

Thanks! super nice library btw!

thanks for the kind words.

Thx for the link, it is interesting to see how the GWT & RxJava teams decided.

I like the strategy of TypeScript to augment untyped JS code with type declarations:

*.ts ---{compile}---> *.js, *.d.ts

Plain JS apps are able to include the compiled .js lib. However, TS apps are also able to use the typed version of the .js lib because of the type declarations *.d.ts.

In fact, it would be a solution to 'clone' all vavr dependencies within the vavr-gwt project:

vavr-gwt
|-- vavr-core-gwt  ---{clones}---> vavr-core
|-- vavr-collection-gwt ---{clones}--> vavr-collection
+-- ...

Then we will not have to use a pre-build step. Instead we only include the GWT compatible libraries. For example we do not clone Vavr's API and Match features.

On top of the clones (see above), there exist one or more commits that are frequently rebased on the clones' master branches. These additional commits manually add the GWT annotations to the sources.

I don't know what would be the best way to release the clone's sources. Maybe a Maven classifier could work? Such as io.vavr:vavr-core:1.0.0-GWT?

Summed up, it would be a clean way to augment Vavr's source code without having to add a pre-build step and/or additional tooling. It is more or less a Maven build configuration change.

What do you think?

[danieldietrich]

Btw, I do not want to produce any extra work. If necessary, I will re-add all the GwtIncompatible annotations, that I removed before, to the clones.

Btw#2, keeping the logic about (GWT) code incompatibility where it is needed (GWT compiler) is a great practice. It is the principle of high cohesion, that will increase the maintainability of vavr-gwt. Vavr-gwt forms a local context. No information should leak out of that context into other modules.

[ibaca]

But the RxJava GWT was not a decision, they don't want to modify its own source code so the last resort was to apply this cloning strategy. The cloning strategy is a nightmare, whatever change you made in the origin project will end up easily as a conflict. And you need to make releases per-version with this conflicting rebased codebase.

And the cohesion is debatable. The @GwtIncompatible is just an annotation, no logic, just a flag that means that this method is compatible or not. So it might be considered "high cohesion" when you put this incompatibility flag in the source that is incompatible. OTOH, if you have a separated module referencing all the other modules specifying which elements are compatible or incompatible, might be considered "low cohesion".

I see this more as the error-prone annotations. So if you are going to use the errorprone annotations to run the errorprone reports, did you put all those annotations in a separated project rebasing all those annotations just because are used with a different purpose. Those annotations describe the source code, and when you as a project decides to use those reports, you also accept to put those annotations in your source code. I see GWT compatibility flags the same way.

Adding those annotations in the original source code make it much easier.

[danieldietrich]

@ibaca

whatever change you made in the origin project will end up easily as a conflict

Conflicts are the best that can happen. If the Vavr core triggers a vavr-gwt build, we will immediately see if s.th. needs to be adapted. The fix can happen in a central way, within the vavr-gwt project.

Prior, the changes needed to be applied within other projects. This was a maintenance hell from the perspective of vavr-gwt.

And the cohesion is debatable.

No, it isn't. Whether it contains logic or not. Piggybacking tool-specific compile-time annotations isn't a choice. We would not have any argument anymore to reject similar annotations needed by other tools/frameworks/libs/whatever.

if you have a separated module referencing all the other modules specifying which elements are compatible or incompatible, might be considered "low cohesion".

I think we need to describe cohesion in a formal way. We seem to have different views on high or low cohesion.

• vavr-gwt does not work anymore, if I remove @GwtIncompatible from vavr-core
• vavr-core does still work, if I remove @GwtIncompatible from vavr-core

This implies, that vavr-gwt depends on @GwtIncompatible. It also implies that vavr-core does not depend on @GwtIncompatible.

'high-cohesion' can be described as a state that keeps things together that depend on each other.

So if you are going to use the errorprone annotations to run the errorprone reports, did you put all those annotations in a separated project rebasing all those annotations just because are used with a different purpose.

Because of that we will not use error-prone annotations. Error-prone is just a tool. I would prefer a tool that does static flow analysis (like Facebook's Flow). Or compiler-plugins that allow to apply user-defined linting rules. It is a gap regarding the general build infrastructure of Java. We will not apply non-standard tooling that acts as a workaround for native Java.

Adding those annotations in the original source code make it much easier.

I agree. But this is where legacy code bases begin...

[ibaca]

I had to try it. Thanks anyway!

[xamde]

I think you can just use any annotation with the local name "@GwtIncompatible", no matter from which package/module it is. So you can create such an annotation in vavr and have no dependency.