noir-lang / noir

Noir is a domain specific language for zero knowledge proofs
https://noir-lang.org
Apache License 2.0
821 stars 177 forks source link

feat: Make macros operate on token streams instead of AST nodes #5301

Closed jfecher closed 1 week ago

jfecher commented 1 week ago

Description

Problem*

Resolves https://github.com/noir-lang/noir/issues/5284

Summary*

Adds a new Quoted type which is the new default output of a quote { ... } expression. In addition, a quote expression now quotes an unparsed, raw vector of tokens terminated by the matching closing brace. This has a number of advantages:

The current plan is to keep the other macro types (Expr, Type, TopLevelItem, etc) as optional outputs of quote only if they're specified after the keyword, e.g. quote Type { Foo<A, B> }. Since the new approach of using token streams by default is much more flexible, there's less of a need for these however.

Additional Context

The formatting of quote expressions has also been fixed. This was a necessary change since these expressions hold token streams now instead of ASTs and thus can't be formatted the old way.

Edit: This does not fix the existing issue with formatting comptime blocks, only quote blocks which were similarly affected.

Documentation*

Check one:

PR Checklist*

jfecher commented 1 week ago

Thought I could get away without a Value::into_tokens method but the macros test is failing without it. I'll need to add it now

jfecher commented 1 week ago

Alright, I managed without it by parsing the new UnquoteMarker token as an expression.