Use incremental parsing

[evhub]

Should enable incremental parsing mode where parses are cached to improve recompilation times for slightly modified files.

[evhub]

The biggest problem with this is figuring out how to effectively store the references to individual grammar items in each item in the parse cache. Though we can certainly pickle and unpickle the cache, doing so creates new grammar objects rather than references to the existing grammar objects, which means the resulting unpickled cache is effectively useless.

What we would need for this is to assign every grammar object a unique, deterministic identifier, pickle and unpickle the identifiers rather than the grammar objects themselves, then look up grammar objects by their identifier when loading in the new cache.

[evhub]

--incremental is now supported as of coconut-develop>=3.0.3-post_dev3, but it needs more testing to verify that it's actually improving compilation times.

[evhub]

Two things that need to be fixed with the current implementation:

• parseIncremental in cPyparsing just shouldn't cache successes, only failures.
• incremental_cache_limit results in only the parse at the end of the file being cached, which is useless the next time we go to parse the file, since the next parse will always start at the beginning.

[evhub]

This works now, but it only provides performance improvements for small files, so it's automatically disabled for large files.

[evhub]

With the new adaptive parsing , this is now providing substantial performance improvements for all files, so it should be enabled by default (and the --incremental option should be removed), though a --no-cache option should be added to disable use of the cache if necessary.