Issue 930 - Using a custom JS generator stops source mapping from working - haxe

[issuesbot]

[Google Issue #930 : https://code.google.com/p/haxe/issues/detail?id=930] by devdavek@googlemail.com, at 15/06/2012, 14:44:34 Using a custom Javascript generator disables any source mapping from being generated when using the -debug flag. Neither the .map file is generated, nor the extra output added to the end of the .js file.

I have reproduced this behaviour in 2.09, plus in today's nightly build (7ba62da42c7d1f9a9e54d5824057b5b72bf0b3a0).

Here is a minimal test case to reproduce:

Main.hx

class Main {
    static function main() {
        trace("Hello world");
        }
    }

compile.hxml

-js bin/debugcase.js -main Main -debug -cp src --macro haxe.macro.ExampleJSGenerator.use()

[issuesbot]

[comment from ncanna...@gmail.com, published at 15/06/2012, 22:30:39] Source mapping works by referencing the lines of the JS output with the lines of the Haxe code. Since a custom code generator will have a custom output, we cannot use the same compile source mapping.

The best would maybe be to have an interface for the mapping engine that would enable to easily reproduce the same output.

Bruno could you suggest an API for it ?

[issuesbot]

[comment from ah.duros, published at 17/06/2012, 08:08:48] As discussed at http://code.google.com/p/haxe/issues/detail?id=697, I'm in favor of deprecating custom JS generators. Making source maps work (and the other JS features that have been added for 2.10) would involve a lot of effort. Not to mention keeping both compilers in step as new features are added.

[issuesbot]

[comment from ncanna...@gmail.com, published at 17/06/2012, 08:47:42] Custom JS generators are necessary to offer a fully customized way to do things such as class registration and splitting .js in several files. I still think there's a lot of valid uses cases for it.

I think that adding source mapping API + hasFeature would not be that hard, and most of the work of the compiler will only affect will only affect expressions anyway.

Also, we don't have to make ExampleJSGenerator as smart / the same as genJs.ml, but it should be potientally possible to do so.

[issuesbot]

[comment from devdavek@googlemail.com, published at 18/06/2012, 11:57:51] If Haxe is being pushed as a statically typed alternative to Javascript then I think there is no option but to allow custom JS generators in order to interface with existing JS libraries. To be honest, I see this as one of Haxe's strongest features with the JS target as it gives pretty much unlimited flexibility.

In my specific case I am implementing a Haxe wrapper for a JS library that expects classes to be declared in a specific way. I believe this would be effectively impossible without custom generators.

var MyClass = Ember.View.extend({
    att35bbab0bd5315eb7eca006de3f6bb1c36fd54fde: "hello",
    att955ac552594ecc04f2b405cb7c2045fec1cbaed3: "world",
    method1: function() { return "hello world"; }
}

var myInstance = MyClass.create({
    method2: function() { return "something else"; }
}

)

I'm sure there are, or will be, examples of other JS frameworks which would require custom JS generation to keep them happy.

I am no expert in the internals of genJs, but I know that I can easily get a 'pos' in the original AST for each custom line of Javascript I am generating; so perhaps instead of using 'print' and 'fprint' to add lines to the buffer I could use api.printJs(codeString:String, pos:Pos), and the source map could be built up in genJs.ml. Then at the end of my custom generate() I could do api.saveSourceMapping() or suchlike.

Just throwing out ideas - apologies if this doesn't fit with the way that the OCaml compiler is constructed :)

[issuesbot]

[comment from devdavek@googlemail.com, published at 18/06/2012, 12:38:56]

697 is closed, so I am making this comment here, but really it follows on from that issue:

I can't conceive of any use case where expression generation would need to be customized. IMO custom JS generation will always be to do with class extension, object instantiation, method generation (i.e. where they are generated). So perhaps a potential middle ground would be to keep the expression generation in OCaml (I assume this is the bulk of it anyway), but defer the structural stuff to a Haxe class which is what is actually used. That way the computationally expensive stuff is still in OCaml, the Haxe class is actually used so there is no need to keep them in sync manually and everything is customizable and out in the open. When just making a little change the dev can extend JSCompiler.hx (or whatever) and just change what they need.

[issuesbot]

[comment from ncanna...@gmail.com, published at 18/06/2012, 13:03:05] @ devdavek : this is already what the custom JD generator does, only the class init / bootstrap is done by the custom generator, the most important part (expression generation) being still handled by genJS.

[issuesbot]

[comment from devdavek@googlemail.com, published at 18/06/2012, 13:34:18] Ah - got mixed up in the other thread; that was a response to @ ah.duros's comment which you had already answered. I guess then my suggestion is the same as your original one; to use a real Haxe class for the structural stuff (porting the source mapping hooks into it too), but I take your point about possible performance hits.

[issuesbot]

[comment from arcn...@gmail.com, published at 16/07/2012, 09:01:16] Are there any conclusions? I'm writing a custom JS generator, and having a way to generate source mapping would be great.

[issuesbot]

[comment from simon.kr...@simn.de, published at 16/07/2012, 09:02:16] I suppose it will be revisited for haxe 3.

[issuesbot]

[comment from ncanna...@gmail.com, published at 11/12/2012, 23:20:42] Up ?

[issuesbot]

[comment from ah.duros, published at 13/12/2012, 04:30:47] I don't think I'll have time to look at it for Haxe 3, to be honest.

[issuesbot]

[comment from si...@haxe.org, published at 21/12/2012, 18:45:17] I could maybe do the implementation if someone suggests an API for it.

[issuesbot]

[comment from si...@haxe.org, published at 22/12/2012, 09:45:36] I don't think having this in haxe code works out. While genExpr in custom generators can call addMapping, the internal recursive calls will not do so.

Wouldn't the solution rather be to expose the spr function of genjs, and let custom generators call that instead of maintaining their own String buffer?

[issuesbot]

[comment from si...@haxe.org, published at 22/12/2012, 10:18:17] Could it work like this?

[issuesbot]

[comment from ncanna...@gmail.com, published at 22/12/2012, 13:06:04] A lot of custom JS also handle different outputs, using their own way to JS into several files.

[issuesbot]

[comment from si...@haxe.org, published at 22/12/2012, 13:09:41] Ok, maybe someone actually using Js/custom generators should work on this. ;)

The only other idea I have is to allow registering a callback which is called instead of add_mapping.

[issuesbot]

[comment from si...@haxe.org, published at 09/02/2013, 19:11:37] No obvious solution at the moment, pushing back to 3.2.

[issuesbot]

[comment from si...@haxe.org, published at 24/02/2013, 17:12:14]

[ncannasse]

Given how little the custom JS generator is used, we will drop this feature.

[nadako]

I'd consider dropping custom js generators in future in favor of onGenerate + macro pf_* configuration

[ncannasse]

@nadako the problem is to maintain an up-to-date reference JS generator that matches the genJS one. And how pf_* is related ? :)

[nadako]

In the sense that onGenerate is quite limited for writing new targets as there's no way to configure compiler behaviour, like pf_static or pf_capture_policy and whatnot.