Esprima.tokenize twice as slow as esprima.parse

google-code-export / esprima

Automatically exported from code.google.com/p/esprima

BSD 2-Clause "Simplified" License

1 stars 0 forks source link

Esprima.tokenize twice as slow as esprima.parse #626

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago

On my setup and for 1.1Mb javascript file, esprima.tokenize(data) 
takes 1300ms, and esprima.parse(data) only 670ms.

This seems confusing, as tokenization should be one of the steps of
parsing (which builds ast).

This might happen due to lots of Array.push calls; in this case, can we have 
SAX interface for tokenization?

Original issue reported on code.google.com by lushni...@chromium.org on 20 Jan 2015 at 5:20

GoogleCodeExporter commented 9 years ago

Let's continue this in https://code.google.com/p/esprima/issues/detail?id=626.

Original comment by ariya.hi...@gmail.com on 9 Feb 2015 at 9:14

Changed state: Migrated

GoogleCodeExporter commented 9 years ago

The correct link for continuation of this: 
https://github.com/jquery/esprima/issues/1019

Original comment by lushni...@chromium.org on 11 Feb 2015 at 1:36