parse a format (see perldoc -f format) as a single token.

reneeb commented 3 years ago

Currently PPI parses formats in a wrong way (see also https://github.com/Perl-Critic/Perl-Critic/issues/917)

Without this fix, this code:

format BYDEPTH_TOP =
Top disk utilization in @*, @* level(s) deep
                        $BASEPATH, $DEPTH
================================================================================

subpath                                                         disk utilization
-------                                                         ----------------
.

sub print_anything {
}

is parsed as

PPI::Document
  PPI::Statement
    PPI::Token::Word    'format'
    PPI::Token::Whitespace      ' '
    PPI::Token::Word    'BYDEPTH_TOP'
    PPI::Token::Whitespace      ' '
    PPI::Token::Operator    '='
    PPI::Token::Whitespace      '\n'
    PPI::Token::Word    'Top'
    PPI::Token::Whitespace      ' '
    PPI::Token::Word    'disk'
    PPI::Token::Whitespace      ' '
    PPI::Token::Word    'utilization'
    PPI::Token::Whitespace      ' '
    PPI::Token::Word    'in'
    PPI::Token::Whitespace      ' '
    PPI::Token::Magic   '@*'
    PPI::Token::Operator    ','
    PPI::Token::Whitespace      ' '
    PPI::Token::Magic   '@*'
    PPI::Token::Whitespace      ' '
    PPI::Token::Word    'level'
    PPI::Structure::List    ( ... ???
      PPI::Statement::Expression
        PPI::Token::Regexp::Substitute      's) deep\n                        $BASEPATH, $DEPTH\n================================================================================\n\nsubpath                                                         disk utilization\n-------                                                         ----------------\n.\n\nsub print_anything {\n}\n'

This completely wrong... With this fix, this is parsed as

PPI::Document
  PPI::Token::Format    'format BYDEPTH_TOP =\nTop disk utilization in @*, @* level(s) deep\n                        $BASEPATH, $DEPTH\n================================================================================\n\nsubpath                                                         disk utilization\n-------                                                         ----------------\n.\n'
  PPI::Token::Whitespace    '\n'
  PPI::Statement::Sub
    PPI::Token::Word    'sub'
    PPI::Token::Whitespace      ' '
    PPI::Token::Word    'print_anything'
    PPI::Token::Whitespace      ' '
    PPI::Structure::Block   { ... }
      PPI::Token::Whitespace    '\n'
  PPI::Token::Whitespace    '\n'

oalders commented 2 years ago

Closing and re-opening to trigger CI.

oalders commented 1 year ago

@wchristian did you have any thoughts on this one?

Perl-Critic / PPI

parse a format (see perldoc -f format) as a single token. #250