Any plans for open sourcing the candies' d.ts -> .java API translator?

[ivmarkov]

(I guess I know what the answer would be but I would still dare to ask.)

Any plans for open sourcing the candies' d.ts -> .java API translator? Or am I missing the elephant in the room and it is already there, in the open?

[renaudpawlak]

You are not the first one to ask :) The API translator will most probably become open source but there is no actual timeline for this. At the moment it is too early. So far, the short term plan is to provide a web service so that anyone will be able to translate its own d.ts. We are working on it. Any comments or suggestions to adapt the candies (structure, APIs) are most welcome.

[ivmarkov]

Thanks for the info and for the very fast answer.

Honestly, the background is that I need something similar, but for GWT, where GWT jsinterop compatible bindings are generated instead of jsweet ones ( https://docs.google.com/document/d/10fmlEYIHcyead_4R1S5wKGs1t2I7Fnp_PaNaa7XTEk0/edit#heading=h.o7amqk9edhb9 ).

... I guess you are aware of the J2CL and "DefinitelyJsInterop" efforts at Google (developed behind closed doors so far, unfortunately... it is such a pity that there is so much duplicated effort).

[renaudpawlak]

I am aware of these GWT 3 efforts. I have been pointed to them a lot. I also find that it is a pity that such efforts are being duplicated. It was a lot of work to make the API translator to work and I am sure that it would have much more impact if the Google folks were willing to take it into account. But it seems that they don't care about these kinds of "details". They probably think that they can do much better than JSweet, which is probably true on the paper.

Now, we have a saying in France that says that "better is the enemy of good". GWT has proven in the past that trying to handle too many issues in the same framework was not better. The JSInterop initiative answers the same needs as JSweet's, however, it is an addon to the existing GWT framework. In JSweet, manipulating and binding to JS program elements is the main purpose and the default mode. In GWT, the default mode is mimicking the Java semantics, but you can bind to JS elements with annotations. This will make two kinds of entities (Java and JS) cohabiting within the same programs, which will probably pose many issues. In JSweet, we follow a rigorous Java vs JS separation so that at the end, we do JS only.

Since adding a syntax layer to JS is the main purpose of JSweet, JSweet has handled many issues in that area, especially by relying on the TypeScript compiler. We have exposed string types, union types, enums, modules and many other features that the TypeScript community has been working on for a while. Through TypeScript, we support ES3 to ES6 and AMD, UMD, Commonjs standards. All this being as simple as lightweight as possible (following the TypeScript design principles), making JSweet very efficient in mapping Java to JS, syntactically speaking. Last but not least, since JSweet goes through TypeScript, it is fully interoperable with TypeScript, which is IMO the right/simplest connection between Java and JavaScript.

Once again, we are open to discussion, integration, and common efforts with the GWT folks, but honestly I am not sure that they would listen to us. In the meantime, we will continue to handle the same issues but following our own pragmatic approach. For instance, we will probably give access to more Java APIs in a total different way. Let's wait and see :)

[ivmarkov]

I absolutely empathy your situation. Google doing their work behind closed doors is just not good for the GWT project, but they do not want to listen. E.g. if their d.ts translator was out there in the open from day one, then probably I would have helped the effort. But Julien did not even bother answering my email - and it is not that I have not contributed a bit to the GWT ecosystem before (sdbg.github.io)...

With the above said, a few small comments / corrections:

Now, we have a saying in France that says that "better is the enemy of good". GWT has proven in the past that trying to handle too many issues in the same framework was not better. The JSInterop initiative answers the same needs as JSweet's, however, it is an addon to the existing GWT framework.

Yes and no: Yes, JSInterop is an add-on to GWT 2.8. Yes, 1:1 the same JSInterop mechanism will exist in J2CL. However, there will be NO GWT 3 in the meantime. Only J2CL, and J2CL is a very different animal from GWT. In fact, if you look closer into the information that leaked about J2CL on the internet, I'm sure you'll recognize jsweet's twin brother. :)

In JSweet, manipulating and binding to JS program elements is the main purpose and the default mode. In GWT, the default mode is mimicking the Java semantics, but you can bind to JS elements with annotations. This will make two kinds of entities (Java and JS) cohabiting within the same programs, which will probably pose many issues. In JSweet, we follow a rigorous Java vs JS separation so that at the end, we do JS only.

What do you mean by Java semantics? Even if I manipulate JS elements, I still would expect e.g. Java generics to work the way they work in Java, or even more importantly, that e.g. the "this" semantics to be that it is lexically bound, and not dynamically bound, like in JS. Smallish differences to Java in the sake of efficient transpilation to JS are OK of course (byte/short/int all mapped too double; overflow vs no overflow etc.).

As for the JSInterop annotations, they just stop the GWT compiler (and I guess will stop the Closure compiler in future if someone wants to postprocess with it the J2CL-generated JS files) from mangling the method names and doing some global optimizations which should not be done when interfacing with existing JS libraries. From my personal experience, they are not that bad at all, especially if annotated Java classes can be generated automatically from d.ts sources. :)

Is the jsweet approach of interfacing with JS all that different form JSInterop? I think it is based on classes with native methods, just like in J2CL + some annotations anyway?

[renaudpawlak]

From what I was able to read on the Internet about J2CL, it is really hard to know exactly what it will do and how it will do it. When Google did its search engine years ago, on the paper it was no different from existing search engines (such as Yahoo's) and everybody was telling them it was stupid to do it. Yet, when they did it, it was the right approach and it was better. I am not saying that JSweet approach will be necessarily better, but it will be different and I am not going to toss away JSweet just because Google is doing something similar "on the paper". At least, we can be sure that JSweet will be different because it goes through TypeScript (this may look as an implementation detail, but having working on JSweet for a while now, you can believe me when I say it is not a detail).

Regarding the JSInterop annotations, well, actually they are not that bad. However, they still imply that there are two kinds of objects: Java (not annotated) and JavaScript (@JSType annotated). I still think that having these two kinds of objects is a bad thing. Besides, I prefer the way we use the @Ambiant annotation in JSweet (or the def package convention) for external declarations (mapped from the way it is done in TypeScript). The namespace and isNative parameters of the @JSType annotations will create a lot of noise and redundant information, which we don't have in JSweet, for instance thanks to the @Root annotation, which defines the JS namespace for a whole package.

That said, one can argue that it is matter of taste. Yet, if I agree with you that it is not really important for automatically-translated definitions, it is more important for the actual code (implementation). It makes me think of JEE (especially early versions of it), which I dislike because there is too much configuration noise that is not handled natively by the IDEs.

What do you mean by Java semantics? Even if I manipulate JS elements, I still would expect e.g. Java generics to work the way they work in Java, or even more importantly, that e.g. the "this" semantics to be that it is lexically bound, and not dynamically bound, like in JS. Smallish differences to Java in the sake of efficient transpilation to JS are OK of course (byte/short/int all mapped too double; overflow vs no overflow etc.).

Yes, we agree I think. Of course, most compilation behavior (including typechecking on generics) is preserved. But for the runtime, it is a matter of balance. JSweet preserves as much as possible Java-like semantics, for the this issue for instance. However, some cases are too heavy to be supported without generating complex code and running into issues. In that too, JSInterop is similar to JSweet (see the section on the instanceof operator and on duplicate names in classes).

Is the jsweet approach of interfacing with JS all that different form JSInterop? I think it is based on classes with native methods, just like in J2CL + some annotations anyway?

So, as I said at the beginning, and to wrap up, it is not that different on the paper, but that does not mean that they are the same. I believe that JSweet still have very cool things to put forward. For instance, we plan to support Java APIs by implementing them in TypeScript, which will make the support of Java API very clean, easily extensible and straightforward (but also not intrusive to JSweet initial philosophy). We are also looking closely at supporting ASM.js-compatible generated code, which could become an important point for JSweet in a close future. Another example: JSweet will support TypeScript annotations soon, which will be important for Angular 2 (btw, I am unsure of how the the JSInterop annotation-based syntax will look when mixing it with TS-level annotations).

Finally, if one day Google releases better quality DefinitelyTyped APIs than JSweet, it will not be a problem for us to support the JSInterop annotations as an alternative way to bridge JavaScript libs. But at this point it is not the case. They are not even releasing anything AFAIK.

[renaudpawlak]

Btw, I was in a hurry when posting my last answer, but I would like to thank you for your insights and helping us to some good pointers. Your opinion is really good to have and it is a very interesting conversation, which helps us to think about what would be the next best move for JSweet. So thank you :D

I understand that your context is to use GWT, but if by chance you had some further ideas for JSweet, I'd be happy to discuss it with you.

[ivmarkov]

The bright side of having multiple implementations of similar ideas is choice. I'm happy to know that if J2CL does not take off, I'm not chained to an obsolete version of GWT forever. :)

As for ideas reg. jsweet, I first have to play with it a bit... Any ideas how to overcome https://github.com/cincheo/jsweet-eclipse-plugin/issues/8 ?

Btw: is the jsweet transpiler also generating V3 sourcemaps?

[renaudpawlak]

As for ideas reg. jsweet, I first have to play with it a bit... Any ideas how to overcome cincheo/jsweet-eclipse-plugin#8 ?

It is on my (very) long todo for this week :)

Btw: is the jsweet transpiler also generating V3 sourcemaps?

Not really. JSweet uses a (dirty) hack to have sourcemaps. Let me explain.

When the sourceMap option is on:

1. JSweet transpiles to TypeScript by preserving the line numbers between the original Java code and the generated TypeScript code. This is quite easy to implement with the prettyprinter, but, of course, it breaks the nice formatting and indentation of generated TypeScript code.
2. JSweet takes the sourcemaps output of tsc and inside the map files, it redirects the TS files to the Java file references, so that the sourcemap files work as if they were generated for the original Java files (it works because of line numbers preservation of step 1).

Of course, it was a hack to have something working faster, but we do not intend to keep it -- it is somewhere in the todo list, but clearly not for this week ;) Better source maps is mentioned as a long term subproject on the wiki page: https://github.com/cincheo/jsweet/wiki. Also, note that when the module option is on, the hack does not work anymore, which fully justifies having a better sourcemap system (see issue #9).

Contributions would be most welcome on that part.

[ivmarkov]

Background of my question was that (once I get jsweet running in Eclipse) I might try to see if I can debug jsweet-generated code with SDBG.