Grammar 'ident' override behaviour

[p6rt]

Migrated from rt.perl.org#120146 (status was 'open')

Searchable as RT120146$

[p6rt]

From @dwarring

The final parse below is failing to match (golfed from CSS​::Grammar)​:

  use v6;   #use Grammar​::Tracer;

  grammar G {

  token nmstrt {\<[_ a..z A..Z]>}   token nmreg {\<[_ \- a..z A..Z 0..9]>+}   token ident {'-'?\\*}   token num {[\+|\-]? (\d* \.)? \d+}

  proto token term { \<...> }   token term​:sym\ {\}   token term​:sym\ {\}

  }

  say G.parse("-my-ident", :rule\); # parsing   use v6;

  grammar G {

  token nmstrt {\<[_ a..z A..Z]>}   token nmreg {\<[_ \- a..z A..Z 0..9]>+}   token ident {'-'?\\*}   token num {[\+|\-]? (\d* \.)? \d+}

  proto token term { \<...> }   token term​:sym\ {\}   token term​:sym\ {\}

  }

  say G.parse("-my-ident", :rule\); # parsing   say G.parse("my-ident", :rule\); # parsing   say G.parse("-my-ident", :rule\); # not parsing

Parse failing on both Parrot and JVM.This has been a problem since about 29th Sept.

Parse succeeds if I uncomment 'use Grammar​::Tracer', ie Heisenbug.

[p6rt]

From @dwarring

Gaah, sorry about the jumble above. Sample case restated below​:

  use v6; ## use Grammar​::Tracer;

  grammar G {

  token nmstrt {\<[_ a..z A..Z]>}   token nmreg {\<[_ \- a..z A..Z 0..9]>}   token ident {['-']?\\*}   token num {[\+|\-]? (\d* \.)? \d+}

  proto token term { \<...> }   token term​:sym\ {\}   token term​:sym\ {\}

  }

  say G.parse("-my-ident", :rule\); # ok   say G.parse("my-ident", :rule\); # ok   say G.parse("-my-ident", :rule\); # failing

[p6rt]

@dwarring - Status changed from 'new' to 'open'

[p6rt]

From @dwarring

I've added a failing (todo) test to S05-grammar/protoregex.t in perl6/roast.

[p6rt]

From @dwarring

I've converted the test case to NQP​:

  # nqp bisection for RT #​120146   # good 62b8cab951394beff10c6d481d63c1f9a945a1cb   # bad 8f719a567c3c4ddba5ef202151fd37ba2ad41355   say("1..1");

  grammar G {

  token nmstrt {\<[ _ a..z ]>}   token nmreg {\<[ _ \- a..z 0..9]>+}   token ident {'-'?\\*}   token num {[\+|\-]?\d+}

  proto token term {*}   token term​:sym\ {\}   token term​:sym\ {\}   }

  my $result := 'nok';   my $exit-status := 1;

  if (G.parse("-my_id", :rule\)) {   $result := 'ok';   $exit-status := 0;   }

  say("$result - term parse");   nqp​::exit($result eq 'ok' ?? 0 !! 1);

...and bisected in NQP​:

commit 9ee3ed7235c9f7110d119ef748cac39a62517e77 Author​: jnthn \jnthn@&#8203;jnthn\.net Date​: Tue Oct 1 00​:07​:51 2013 +0200

  Treat ident declaratively.  
  timotimo++ and diakopter++

Aha. with that commit, ident is now treated more declatively, so re-declaring ident has become problematic.

[p6rt]

From @dwarring

I've added another test to S05-grammar/protoregex.t for the following variation - 'ident' as an alias​:

say("1..1");

grammar G {

  token nmstrt {\<[ _ a..z ]>}   token nmreg {\<[ _ \- a..z 0..9]>+}   token my-id {'-'?\\*}   token num {[\+|\-]?\d+}

  proto token term {*}   token term​:sym\ {\}   token term​:sym\ {\} }

my $result := 'nok'; my $exit-status := 1;

if (G.parse("-my_id", :rule\)) {   $result := 'ok';   $exit-status := 0; }

say("$result - term parse"); nqp​::exit($result eq 'ok' ?? 0 !! 1);

[dwarring]

Maybe the built-in rules should('ve) be uppercase anyway, e.g. "abc" ~~ /<IDENT>/