search

goccy / p5-Compiler-Lexer

Lexical Analyzer for Perl5
Other
46 stars 17 forks source link

RegOpt of `m` has been misrecognized as match operator #43

Closed moznion closed 9 years ago

moznion commented 10 years ago

Code to reproduce

Compiler::Lexer->new->tokenize(<<'...');
/foo/m;
/bar/;
...

Result

[
  bless( {
    data => "/",
    has_warnings => 0,
    kind => 24,
    line => 1,
    name => "RegDelim",
    stype => 0,
    type => 151
  }, 'Compiler::Lexer::Token' ),
  bless( {
    data => "foo",
    has_warnings => 0,
    kind => 24,
    line => 1,
    name => "RegExp",
    stype => 0,
    type => 186
  }, 'Compiler::Lexer::Token' ),
  bless( {
    data => "/",
    has_warnings => 0,
    kind => 24,
    line => 1,
    name => "RegDelim",
    stype => 0,
    type => 151
  }, 'Compiler::Lexer::Token' ),
  bless( {
    data => "m",
    has_warnings => 0,
    kind => 29,
    line => 1,
    name => "RegOpt",
    stype => 0,
    type => 144
  }, 'Compiler::Lexer::Token' ),
  bless( {
    data => ";",
    has_warnings => 0,
    kind => 24,
    line => 1,
    name => "RegDelim",
    stype => 0,
    type => 151
  }, 'Compiler::Lexer::Token' ),
  bless( {
    data => "\n/bar/",
    has_warnings => 0,
    kind => 24,
    line => 2,
    name => "RegExp",
    stype => 0,
    type => 186
  }, 'Compiler::Lexer::Token' ),
  bless( {
    data => ";",
    has_warnings => 0,
    kind => 24,
    line => 2,
    name => "RegDelim",
    stype => 0,
    type => 151
  }, 'Compiler::Lexer::Token' )
]

The option of regular expression of m has been misrecognized as match operator (even if it is handled as RegOpt).

In this case, this m becomes m;;, and semicolons are treated as RegDelim. I'm expecting as follows;