Support for source maps.

[radare]

When an there's a wrong construction like that does not compiles only the generated javascript code is shown, but not the wisp information, so error messages are not really userfriendly.

[Gozala]

Yeap! This requires some work. If you could also post example that would be great, thanks!

[radare]

Here's an example:

• error shows that the problem is in line 3, which is wrong (it's line 2)
• it displays the line of generated javascript that failed, but there's no reference to the wisp code
• 'has no method error' just clarifies the problem
• this is a simple example, but with more complex constructions error messages are not that clear. i can post other tests if you like

$ cat error.wisp 
(.log console "Starting crash...")
(.error (require "http"))

$ wisp error.wisp 
/Users/pancake/prg/wisp/www/error.wisp:3
on (exports, require, module, __filename, __dirname) { require("http").error()
                                                                       ^
TypeError: Object #<Object> has no method 'error'
    at Object.<anonymous> (/Users/pancake/prg/wisp/www/error.wisp:1:79)
    at Module._compile (module.js:449:26)
    at Object.require.extensions..wisp (/opt/local/lib/node_modules/wisp/lib/engine/node.js:16:17)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at main (/opt/local/lib/node_modules/wisp/lib/wisp.js:58:12)
    at Object.<anonymous> (/opt/local/lib/node_modules/wisp/bin/wisp.js:2:23)
    at Module._compile (module.js:449:26)
    at Object.Module._extensions..js (module.js:467:10)
    at Module.load (module.js:356:32)

[Gozala]

I'm running into these issues myself when working on compiler itself. Unfortunately I don't quite have idea about how I can carry over lineinfo into runtime errors. Probably source maps could help here, but they are not yet widely implemented neither node has support for it.

If you have some ideas on that I'd love to listen.

[krisajenkins]

I've made some headway on this, if you're interested. The node-source-map project makes it possible to create & use source maps from node. Getting the the source line & column is fairly easy, because it's there in the metadata. I'm just finding it a bit messy to try & generate the output line & column.

This is where I've got to so far: https://github.com/krisajenkins/wisp/commit/6cf5a179998133c48ba86ac2422039ac3e790f61

Will gladly help out further...

[Gozala]

I've made some headway on this, if you're interested. The node-source-map project makes it possible to create & use source maps from node.

Oh wow this is really cool, I'm really excited now!!

Getting the the source line & column is fairly easy, because it's there in the metadata. I'm just finding it a bit messy to try & generate the output line & column.

Yeah I know :( Main problem is that compile step both does macro-expansion and compiling into JS. I was actually working on changing that to a two step porcess 1. macro-expansion + desugaring 2. compiling results to a host language.

This is where I've got to so far: krisajenkins@6cf5a17

I'm surprised you could even get even that far with a state of compiler now.

Will gladly help out further...

If you'll have some time to hack on this that would be amazing! I find myself little occupied lately, and was not able to spend much time on this, but I'd be more than happy to assist you in any way I can.

[krisajenkins]

Just to keep this thread alive, I'm still looking at it. :-)

One thought - I find the Makefile really painful. If I put together a Grunt file, for builds, would you want it?

[Gozala]

Just to keep this thread alive, I'm still looking at it. :-)

I should have probably posted here, I'm working on the branch that would compile to JS AST that can be later compiled using escodegen. With that in place adding source maps should be extremely easy I think.

One thought - I find the Makefile really painful. If I put together a Grunt file, for builds, would you want it?

I would really like to avoid dependency on Grunt or other less common Make replacements. Maybe there's a way we could fix the pain without switching to other tools ?

[krisajenkins]

Heh, I don't think there's any real way to fix the pain of Make without a time-machine... Ne'er mind. I'm glad I asked first.

I look forward to seeing the results of escodegen. :-D

[Gozala]

@krisajenkins I just wanted to reach you out to tell that escodegen work #52 is going greatly and simple forms already can be generated with source-maps, I hope to finish up version with source map support next weekend ;)

[krisajenkins]

Cool! I look forward to it.

Oh, you might find something like continuation.js interesting. I'll go over an escodegen-ready tree and give you extra TCOs for free.

[Gozala]

So it did not end up as simple as I anticipated, but initial support is there now. I also made small video showing chrome debugger working with wisp: http://youtu.be/91fzvASqgII

There are still lot's of polishing work to be done to make sure all forms can be mapped, but this may be a good opportunity to get involved.

In nearest future I'll replace old compiler with a new one.

[shaunxcode]

have you created specific issues for the forms which need to be mapped?

[Gozala]

@shaunxcode well there is an issue with primitive forms that prevents source mapping (see #59) although I don't think it's that big of an issue, as debugger on primitives is not all that useful.

I don't have issues for other cases, since I'm basically learning about them by testing & then fixing them. But you're making a good point I'll make sure to create issues when I discover them and will comment if I'm working on them.

BTW most of the source mapping issues are caused by additional code generation during syntax expansion face, cause new form don't really map to anything. Mostly it's matter of just adding a metadata to an added forms.

Although sometimes it's issues caused by escodegen, I had to do bunch of fixes to it already and have reported few others that are not that critical.