This PR overhauls comment handling to support nested block comments. This was done by wrapping Gramatika's out-of-the-box ParseStream with an implementation that intercepts all peek, check and consume calls to check for comment-start tokens.
Comment nodes are stored in a separate buffer on this wrapper type that can be accessed by calling wgsl_parser::ParseStream::into_inner after parsing the syntax tree.
To accommodate error-handling while trying to parse comments, parse errors have also been moved to a separate buffer, which is also available on the type returned from into_inner.
Breaking Changes
Previously, the following two lines of code were equivalent:
let tree: SyntaxTree = gramatika::ParseStream::from(input).parse()?;
let tree: SyntaxTree = wgsl_parser::ParseStream::from(input).parse()?;
That's because wgsl_parser::ParseStream was just a type alias. That is no longer the case, so import paths may need to be updated to ensure ParseStream is being imported from wgsl_parser, notgramatika.
ParseStream::into_inner() now returns a ParseResult instead of a tuple
Token::BlockComment and Token::LineComment variants have been removed
Token::CommentStart and Token::CommentEnd variants have been added
This PR overhauls comment handling to support nested block comments. This was done by wrapping Gramatika's out-of-the-box
ParseStream
with an implementation that intercepts allpeek
,check
andconsume
calls to check for comment-start tokens.Comment nodes are stored in a separate buffer on this wrapper type that can be accessed by calling
wgsl_parser::ParseStream::into_inner
after parsing the syntax tree.To accommodate error-handling while trying to parse comments, parse errors have also been moved to a separate buffer, which is also available on the type returned from
into_inner
.Breaking Changes
That's because
wgsl_parser::ParseStream
was just a type alias. That is no longer the case, so import paths may need to be updated to ensureParseStream
is being imported fromwgsl_parser
, notgramatika
.ParseStream::into_inner()
now returns aParseResult
instead of a tupleToken::BlockComment
andToken::LineComment
variants have been removedToken::CommentStart
andToken::CommentEnd
variants have been addedDecl::Error
variant has been removedStmt::Error
variant has been removedExpr::Error
variant has been removed