JuliaLang / JuliaSyntax.jl

The Julia compiler frontend
Other
272 stars 35 forks source link

eliminate dependency on Base.is_id_char and julia-specific utf8proc #377

Open stevengj opened 11 months ago

stevengj commented 11 months ago

As discussed in #372, in order to make the JuliaSyntax parsing independent of the version of Julia in which it runs, it needs to:

  1. Add a dependency on utf8proc_jll, rather than using the utf8proc bundled with Julia. This is important because the supported version of Unicode (for character classification) depends on utf8proc.
  2. Re-implement Base.is_id_start_char and Base.is_id_char to identify which characters can appear in identifiers (or technically, which characters can begin graphemes, now that #372 is merged … this could simplify the rules slightly since we no longer need to explicitly check for combining marks).
c42f commented 11 months ago

Thanks for outlining what needs to be done here :)

Just wondering - do you think you'll have bandwidth to work on this yourself or should I or someone else take it on? I feel like tackling it relatively soon because I'd like to be able to bump JuliaSyntax version which is used in Base, even on Julia-1.10.x without introducing new features. And I belated realized that #372 breaks the ability to do that.

stevengj commented 11 months ago

I'm working on it.