microsoft / tolerant-php-parser

An early-stage PHP parser designed for IDE usage scenarios.
MIT License
883 stars 80 forks source link

API should support getting trailing trivia #90

Open mousetraps opened 7 years ago

mousetraps commented 7 years ago

This would be simple once #88 is implemented (trailing trivia == leading trivia of the next token)

mousetraps commented 7 years ago

Also, eventually we could get more fancy and move towards a definition similar to Roslyn, where each token's trailing comments/whitespace is all the trailing comments/whitespace up until the first newline. And this would be where the leading comments/whitespace for the next token begins.

DanielRosenwasser commented 7 years ago

For some more context on this: https://github.com/Microsoft/TypeScript/wiki/Architectural-Overview#terminology:

We follow Roslyn's notion of trivia ownership for comment ownership. In general, a token owns any trivia after it on the same line up to the next token. Any comment after that line is associated with the following token. The first token in the source file gets all the initial trivia, and the last sequence of trivia in the file is tacked onto the end-of-file token, which otherwise has zero width.

For most basic uses, comments are the "interesting" trivia. The comments that belong to a Node which can be fetched through the following functions:

Function Description
ts.getLeadingCommentRanges Given the source text and position within that text, returns ranges of comments between the first line break following the given position and the token itself (probably most useful with ts.Node.getFullStart).
ts.getTrailingCommentRanges Given the source text and position within that text, returns ranges of comments until the first line break following the given position (probably most useful with ts.Node.getEnd).