Ruby parser is not valid for modern ruby versions

[le0pard]

Parser generator for ruby generate code with _1 and _2, which is reserved in latest ruby versions:

Added from ruby 2.7 - https://www.bigbinary.com/blog/ruby-2-7-introduces-numbered-parameters-as-default-block-parameters

Steps to reproduce

Ruby version:

ruby -v
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [arm64-darwin23]

Create calc-ast.g file with content:

%lex

%%

\s+                             /* skip whitespace */
^(?:\d+(?:\.\d*)?|\.\d+)        return 'NUMBER'
\w+                             return 'ID'

/lex

%{
  const functionList = ['sin', 'cos', 'tan'];

  function FunctionExpression(id, value) {
    if (functionList.includes(id)) {
      return {
        type: 'Function',
        name: id,
        value
      }
    }
  }

  function BinaryExpression(operator, left, right) {
    return {
      type: 'BinaryExpression',
      operator,
      left,
      right
    }
  }

  function UnaryExpression(value) {
    return {
      type: 'UnaryExpression',
      value
    }
  }

  function NumericLiteral(value) {
    return {
      type: 'Number',
      value: Number(value)
    }
  }
%}

%left '+' '-'
%left '*' '/'
%right '^'

%%

e
  : e '+' e    { $$ = BinaryExpression($2, $1, $3) }
  | e '-' e    { $$ = BinaryExpression($2, $1, $3) }
  | e '*' e    { $$ = BinaryExpression($2, $1, $3) }
  | e '/' e    { $$ = BinaryExpression($2, $1, $3) }
  | e '^' e    { $$ = BinaryExpression($2, $1, $3) }
  | '(' e ')'  { $$ = $2 }
  | ID '(' e ')'  { $$ = FunctionExpression($1, $3) }
  | '-' e      { $$ = UnaryExpression($2) }
  | NUMBER     { $$ = NumericLiteral($1) }
  ;

Generate parser:

syntax-cli -g calc-ast.g -m lalr1 -o calc-ast-parser.rb

Now try to use it:

irb(main):001> require './calc-ast-parser'
<internal:/Users/leo/.asdf/installs/ruby/3.3.0/lib/ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:127:in `require': Search timed out SYNTAX_SUGGEST_TIMEOUT=1, run with SYNTAX_SUGGEST_DEBUG=1 for more info/Users/leo/.asdf/installs/ruby/3.3.0/lib/ruby/3.3.0/timeout.rb:43:in `rescue in handle_timeout'
/Users/leo/.asdf/installs/ruby/3.3.0/lib/ruby/3.3.0/timeout.rb:40:in `handle_timeout'
/Users/leo/.asdf/installs/ruby/3.3.0/lib/ruby/3.3.0/timeout.rb:195:in `timeout'
/Users/leo/Downloads/sqlite_test/calc-ast-parser.rb:94: _1 is reserved for numbered parameter (SyntaxError)
/Users/leo/Downloads/sqlite_test/calc-ast-parser.rb:98: _1 is reserved for numbered parameter
/Users/leo/Downloads/sqlite_test/calc-ast-parser.rb:98: _2 is reserved for numbered parameter
/Users/leo/Downloads/sqlite_test/calc-ast-parser.rb:98: _3 is reserved for numbered parameter
/Users/leo/Downloads/sqlite_test/calc-ast-parser.rb:102: _1 is reserved for numbered parameter
/Users/leo/Downloads/sqlite_test/calc-ast-parser.rb:102: _2 is reserved for numbered parameter
/Users/leo/Downloads/sqlite_test/calc-ast-parser.rb:102: _3 is reserved for numbered parameter
/Users/leo/Downloads/sqlite_test/calc-ast-parser.rb:106: _1 is reserved for numbered parameter
/Users/leo/Downloads/sqlite_test/calc-ast-parser.rb:106: _2 is reserved for numbered parameter
/Users/leo/Downloads/sqlite_test/calc-ast-parser.rb:106: _3 is reserved for numbered parameter
/Users/leo/Downloads/sqlite_test/calc-ast-parser.rb:110: _1 is reserved for numbered parameter
/Users/leo/Downloads/sqlite_test/calc-ast-parser.rb:110: _2 is reserved for numbered parameter
/Users/leo/Downloads/sqlite_test/calc-ast-parser.rb:110: _3 is reserved for numbered parameter
/Users/leo/Downloads/sqlite_test/calc-ast-parser.rb:114: _1 is reserved for numbered parameter
/Users/leo/Downloads/sqlite_test/calc-ast-parser.rb:114: _2 is reserved for numbered parameter
/Users/leo/Downloads/sqlite_test/calc-ast-parser.rb:114: _3 is reserved for numbered parameter
/Users/leo/Downloads/sqlite_test/calc-ast-parser.rb:118: _1 is reserved for numbered parameter
/Users/leo/Downloads/sqlite_test/calc-ast-parser.rb:118: _2 is reserved for numbered parameter
/Users/leo/Downloads/sqlite_test/calc-ast-parser.rb:118: _3 is reserved for numbered parameter
/Users/leo/Downloads/sqlite_test/calc-ast-parser.rb:122: _1 is reserved for numbered parameter
/Users/leo/Downloads/sqlite_test/calc-ast-parser.rb:122: _2 is reserved for numbered parameter
/Users/leo/Downloads/sqlite_test/calc-ast-parser.rb:122: _3 is reserved for numbered parameter
/Users/leo/Downloads/sqlite_test/calc-ast-parser.rb:122: _4 is reserved for numbered parameter
/Users/leo/Downloads/sqlite_test/calc-ast-parser.rb:126: _1 is reserved for numbered parameter
/Users/leo/Downloads/sqlite_test/calc-ast-parser.rb:126: _2 is reserved for numbered parameter
/Users/leo/Downloads/sqlite_test/calc-ast-parser.rb:130: _1 is reserved for numbered parameter
/Users/leo/Downloads/sqlite_test/calc-ast-parser.rb:311: syntax error, unexpected '{', expecting `then' or ';' or '\n'
...f (functionList.includes(id)) {
...                              ^
/Users/leo/Downloads/sqlite_test/calc-ast-parser.rb:323: syntax error, unexpected ',', expecting =>
      operator,
              ^
/Users/leo/Downloads/sqlite_test/calc-ast-parser.rb:332: syntax error, unexpected '\n', expecting =>
      value
           ^
/Users/leo/Downloads/sqlite_test/calc-ast-parser.rb:334: syntax error, unexpected '}', expecting `end' or dummy end
/Users/leo/Downloads/sqlite_test/calc-ast-parser.rb:417: target of repeat operator is not specified: /* skip whitespace */
/Users/leo/Downloads/sqlite_test/calc-ast-parser.rb:597: class/module name must be CONSTANT
class calc-ast-parser < YYParse; end
      ^~~~
/Users/leo/Downloads/sqlite_test/calc-ast-parser.rb:597: syntax error, unexpected '-'
class calc-ast-parser < YYParse; end
          ^
/Users/leo/Downloads/sqlite_test/calc-ast-parser.rb:597: syntax error, unexpected end-of-input, expecting `end' or dummy end
... calc-ast-parser < YYParse; end
...                               ^

    from <internal:/Users/leo/.asdf/installs/ruby/3.3.0/lib/ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:127:in `require'
    from (irb):1:in `<main>'
    from <internal:kernel>:187:in `loop'
    from /Users/leo/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/irb-1.12.0/exe/irb:9:in `<top (required)>'
    from /Users/leo/.asdf/installs/ruby/3.3.0/bin/irb:25:in `load'
    from /Users/leo/.asdf/installs/ruby/3.3.0/bin/irb:25:in `<main>'

With js parser all good:

> const Parser = require('./calc-ast-parser')
undefined
> Parser
{
  setOptions: [Function: setOptions],
  getOptions: [Function: getOptions],
  parse: [Function: parse],
  setTokenizer: [Function: setTokenizer],
  getTokenizer: [Function: getTokenizer],
  onParseBegin: [Function: onParseBegin],
  onParseEnd: [Function: onParseEnd],
  onShift: [Function: onShift]
}
> Parser.parse('1 + 2 * 3')
{
  type: 'BinaryExpression',
  operator: '+',
  left: { type: 'Number', value: 1 },
  right: {
    type: 'BinaryExpression',
    operator: '*',
    left: { type: 'Number', value: 2 },
    right: { type: 'Number', value: 3 }
  }
}

P.S. Example in README for ruby incorrect, need to be require 'CalcParser', not require 'CalcParser.php' (not php)

[DmitrySoshnikov]

@le0pard, thanks for the report. Please feel free to submit a PR to fix the underscores issue in variable names (and check it's compatible with previous Ruby versions), however the issue you're having here also relates to JS syncs in the module prologue.

/Users/leo/Downloads/sqlite_test/calc-ast-parser.rb:311: syntax error, unexpected '{', expecting `then' or ';' or '\n'
...f (functionList.includes(id)) {

This is now a valid Ruby syntax, but the syntax from JS.

For Ruby example please address this example file. It can be converted from JSON grammar format to Bison/Yacc format, but you need to use Ruby syntax when define extra functions, etc.