search

cs-au-dk / Artemis

automated testing for JavaScript
http://www.brics.dk/artemis/
GNU General Public License v3.0
36 stars 10 forks source link

Prettify JavaScript code automatically #39

Closed sema closed 11 years ago

sema commented 11 years ago

Minified and auto generated JavaScript is very difficult to inspect in our coverage reports. Prettify all JavaScript as early as possible in WebKit.

BenSpencer commented 11 years ago

A couple of notes from my initial searching:

sema commented 11 years ago

So our solution would be to use a proxy. Have you looked into this or should I take a stab at it?

BenSpencer commented 11 years ago

Yes I have been looking at this problem this morning. I don't actually think that modifying the SourceProvider will be too bad (although I was about to test it so maybe I will change my mind!).

As for a C++-based prettifier, I haven't had much luck. I have looked at quite a lot and our options seem to be either a JS prettifier written in either JS or Python (jsbeautifier.org seems one of the best/most robust), or alternatively a C-like language beautifier not officially supporting JS but which is written in C++ (AStyle seems to be the most amenable to formatting JS). The output of AStyle is not as good as the "true" JS beautifiers, but it is extremely customisable, so maybe we can set up some configuration which looks really good for it (and it is OK by default).

How do you feel about this? Is adding some C++/python binding feasible, or even a call to an external CLI app?

sema commented 11 years ago

I have conducted a quick test, applying AStyle and jsbeautifier.org to a minified version of jQuery. The result given by AStyle is very poor compared to jsbeautifier (transforming jQuery into a 250 line program, far from the thousands of lines it should be uncompressed).

If these are our candidates then I would suggest we focus on jsbeautifier.

BenSpencer commented 11 years ago

Yes I agree unless there is a big problem with including some binding to some external (non C++) code or I can come up with some very good configuration for AStyle (which I haven't tried and I'm a bit skeptical of being able to).

Also, I have been trying some modifications to SourceProvider.h and SourceCode.h in JavaScriptCore/parser, but none of my changes have been showing up after compilation. Even calls to exit() are ignored, so I think my changes are simply not being used. Is there any trick I am supposed to be using to modify Webkit files?

sema commented 11 years ago

Not if you are modifying existing files (adding new files requires some additional steps).

One possible cause could be confusion on Debug and Release compilations, where Artemis only uses one of them (depending on your LD_LIBRARY_PATH).

sema commented 11 years ago

In theory we should be able to use WebKit's own JavaScript interpreter to execute the JS version of jsbeautifier. In practice this could prove difficult though.

BenSpencer commented 11 years ago

Yes that seems like it might be ideal if you think we could do it.

if we do go down the JS route, the webkit inspector itself seems to use this file: WebKit/Source/WebCore/inspector/front-end/JavaScriptFormatter.js

sema commented 11 years ago

We could try to identify where the inspector uses this file, that should give us a template for doing the same ourselves.

On 11 April 2013 16:00, Ben Spencer notifications@github.com wrote:

Yes that seems like it might be ideal if you think we could do it.

if we do go down the JS route, the webkit inspector itself seems to use this file: WebKit/Source/WebCore/inspector/front-end/JavaScriptFormatter.js

— Reply to this email directly or view it on GitHubhttps://github.com/cs-au-dk/Artemis/issues/39#issuecomment-16235979 .

BenSpencer commented 11 years ago

As I understand it (and please correct me if I'm wrong...) the whole of the Inspector is a JavaScript app running on top of the page, which connects back to the "server" part of the main browser to get the information to display. So I agree that this is a good place to look, there must be some component which loads the front-end JavaScript app.

Do you think it will be part of the whole front-end "chrome"? Is that implemented as some sort of XML document?

I should also clarify: when I said "the inspector uses this file" I really mean "this file exists"!

sema commented 11 years ago

I actually don't know much about the inspector (besides the fact that we include its compile flag in our makefile). I suspect that it is implemented as a standard chrome extension (in HTML and JavaScript), and if that is the case then we are probably looking at a very standardized interface for running the inspector.

On 11 April 2013 17:31, Ben Spencer notifications@github.com wrote:

As I understand it (and please correct me if I'm wrong...) the whole of the Inspector is a JavaScript app running on top of the page, which connects back to the "server" part of the main browser to get the information to display. So I agree that this is a good place to look, there must be some component which loads the front-end JavaScript app.

Do you think it will be part of the whole front-end "chrome"? Is that implemented as some sort of XML document?

I should also clarify: when I said "the inspector uses this file" I really mean "this file exists"!

— Reply to this email directly or view it on GitHubhttps://github.com/cs-au-dk/Artemis/issues/39#issuecomment-16241994 .

BenSpencer commented 11 years ago

Hey, just to keep you updated. I have been making some modifications to SourceCode::makeSource() and SourceProvider::getRange() to try to find an appropriate place to inject some modified JS. Looking at the source which those functions are handling, I think it has already been processes somewhere. I will try to see if I can find where this source comes from. I think the best place for our beautification would be even earlier in the parser than this.

Also, although I have been able to do some crude injections from here, it is not affecting the coverage report. Where does the coverage report get it’s information about what the page’s Javascript is from?

Thanks, Ben

sema commented 11 years ago

See the javascript_code_loaded function in QWebExecutionListener, As far as I know, it is the source code observed from this call which is used for the coverage report.

sema commented 11 years ago

I have poked a bit into the inspector module. It seems the inspector is implemented mainly in JavaScript, with a mixture of python scripts and C++ files. So I have concluded that it will take some time to identify how we can invoke the prettifier from within WebKit.

If you have not solved the "source provider" part yet, then I will suggest a change of strategy. I have started to modify an old proxy, and it will probably only require a couple of hours to modify it for pretty-printing. This would be better for now, such that we can get past this tool building stuff.

BenSpencer commented 11 years ago

Maybe the proxy is the best way to go, then…

I haven’t made any more progress, but I also haven’t spent much time at all on it this week either I’m sad to say. I did look into tracing forwards from where the page is initially loaded but I did not manage to find where the javascript files and code is split out and managed.

budde377 commented 11 years ago

I found this project: http://jsbeautifier.org/ . Maybe it could be of some help?

sema commented 11 years ago

I'm looking at their node.js library as we speak :)