search

julia-vscode / CSTParser.jl

A concrete syntax tree parser for Julia
Other
105 stars 35 forks source link

Type instability in CSTParser #325

Open ChenNingCong opened 2 years ago

ChenNingCong commented 2 years ago

Hi! I am working on a static compiler prototype for Julia (as a student project, check this branch: link.) It's based on LLVM's new linker JITLink and can produce binary (so to reduce latency completely). I test it on Tokenize and CSTParser because they are just like a static C library and every function is well typed. The result is promising. The only latency left is the latency to loading precompilation cache (.ji) of the packages, because function definitions and type definitions are not cached in the form of binary code.

However I identified some type instabilities in this library and they are hard to get compiled right now. Type instability will generate abstract types or partially applied types, which are currently unhandled in my prototype. Anyway, fixing them can improve compilation speed and precompilation cache, even if we don't want to compile CSTParser statically.

There are mainly two sources of instabilities:

  1. Functional programming, for example the has_error in src/util.jl. It calls any(has_error, x.trivia) and doesn't specialize on has_error and leads to a generic call. This may due to that has_error has two methods.

    Another example is the broadcast operator of to_codeobject in src/conversion.jl. Because to_codeobject returns Any,to_codeobject.(x.args)... will be inferred as AbstractVector, not Vector{Any}. Here we should

  2. The Union{Nothing, ...} field in EXPR (trivia, head). Many function directly access these fields and they don't test . Beside, EXPR is mutable, so even if we have asserted that x.args isn't nothing, the following use of x.args is still type unstable, because x.args may change. A good news is that this doesn't block static compiling, because union splitting works pretty fine and we only need to generate a runtime type check.

If needed, I'd like to make PRs to improve type instability in CSTParser. What's your opinion about this improvement?

davidanthoff commented 2 years ago

I think such contributions would be fantastic. One long term goal of ours is to be able to compile the entire language server into a self-contained (small!) binary and ship that with the Julia VS Code extension, so that things work even if the user doesn't have Julia installed at all, and also to help with latency. Plus, in general, getting rid of type instabilities is always good :) So, PRs are welcome, I'd say :)