Overriding terminal indirectly through multiple module imports fails when using separator repetition

[Tiger-Tom]

• parglare version: 0.18.0
• Python version: 3.12.2
• Operating System: Linux 6.8.7-arch1-1

Description

When overriding a terminal through multiple layers of grammar (base.sub.SEP), all rules using separator repetition syntax with the overridden terminal as the separator will fail with a KeyError whilst constructing Grammar

What I Did

Say I have three files:

• base.pg, which imports sub.pg and defines a rule utilizing sub.Rule
• sub.pg, which defines a terminal SEP and a rule Rule that uses +[SEP]
• overrider.pg, which imports base.pg and defines a terminal base.sub.SEP

Example files

base.pg

import './sub.pg' ;
main: sub.Rule+ ;

sub.pg

Rule:  ITEM+[SEP] ;
terminals
ITEM: /\w+/ ;
SEP:  '.' ;

overrider.pg

import './base.pg' ;
main: base.main ;
terminals
base.sub.SEP: ',' ;

When constructing grammar from base.pg, everything is fine.
However, upon constructing grammar from overrider.pg: parglare.exceptions.GrammarError: Error at /home/shae/Desktop/Coding/Caustic/CausticGrammar/src/caustic/grammar/test/sub.pg:1:7:"Rule: **> ITEM+[SEP]" => Unexisting module "ITEM_1_base" in reference "ITEM_1_base.sub.SEP"

Full stack trace

Traceback (most recent call last):
  File "/home/shae/.local/lib/python3.12/site-packages/parglare/grammar.py", line 663, in resolve_symbol_by_name
    return self.symbols_by_name[symbol_fqn]
KeyError: 'base.sub.ITEM_1_base.sub.SEP'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/shae/.local/lib/python3.12/site-packages/parglare/grammar.py", line 663, in resolve_symbol_by_name
    return self.symbols_by_name[symbol_fqn]
KeyError: 'sub.ITEM_1_base.sub.SEP'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/shae/.local/lib/python3.12/site-packages/parglare/grammar.py", line 663, in resolve_symbol_by_name
    return self.symbols_by_name[symbol_fqn]
KeyError: 'ITEM_1_base.sub.SEP'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/shae/.local/lib/python3.12/site-packages/parglare/grammar.py", line 668, in resolve_symbol_by_name
    imported_pg_file = self.imports[import_module_name]
KeyError: 'ITEM_1_base'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/shae/Desktop/Coding/Caustic/CausticGrammar/src/caustic/grammar/test/test.py", line 14, in 
    parglare.Grammar.from_file('./overrider.pg')
  File "/home/shae/.local/lib/python3.12/site-packages/parglare/grammar.py", line 1159, in from_file
    return Grammar._parse('parse_file', file_name, **kwargs)
  File "/home/shae/.local/lib/python3.12/site-packages/parglare/grammar.py", line 1138, in _parse
    g = Grammar(productions=productions,
  File "/home/shae/.local/lib/python3.12/site-packages/parglare/grammar.py", line 743, in __init__
    self._init_grammar()
  File "/home/shae/.local/lib/python3.12/site-packages/parglare/grammar.py", line 757, in _init_grammar
    self._add_resolve_all_production_symbols()
  File "/home/shae/.local/lib/python3.12/site-packages/parglare/grammar.py", line 802, in _add_resolve_all_production_symbols
    add_productions(list(self.productions))
  File "/home/shae/.local/lib/python3.12/site-packages/parglare/grammar.py", line 797, in add_productions
    add_productions(rhs_elem.productions)
  File "/home/shae/.local/lib/python3.12/site-packages/parglare/grammar.py", line 797, in add_productions
    add_productions(rhs_elem.productions)
  File "/home/shae/.local/lib/python3.12/site-packages/parglare/grammar.py", line 797, in add_productions
    add_productions(rhs_elem.productions)
  File "/home/shae/.local/lib/python3.12/site-packages/parglare/grammar.py", line 784, in add_productions
    self._resolve_ref(rhs_elem)
  File "/home/shae/.local/lib/python3.12/site-packages/parglare/grammar.py", line 837, in _resolve_ref
    symbol = self.resolve_symbol_by_name(symbol_name,
  File "/home/shae/.local/lib/python3.12/site-packages/parglare/grammar.py", line 674, in resolve_symbol_by_name
    return imported_pg_file.resolve_symbol_by_name(name, location)
  File "/home/shae/.local/lib/python3.12/site-packages/parglare/grammar.py", line 1231, in resolve_symbol_by_name
    return self.pgfile.resolve_symbol_by_name(symbol_name, location)
  File "/home/shae/.local/lib/python3.12/site-packages/parglare/grammar.py", line 674, in resolve_symbol_by_name
    return imported_pg_file.resolve_symbol_by_name(name, location)
  File "/home/shae/.local/lib/python3.12/site-packages/parglare/grammar.py", line 1231, in resolve_symbol_by_name
    return self.pgfile.resolve_symbol_by_name(symbol_name, location)
  File "/home/shae/.local/lib/python3.12/site-packages/parglare/grammar.py", line 670, in resolve_symbol_by_name
    raise GrammarError(
parglare.exceptions.GrammarError: Error at /home/shae/Desktop/Coding/Caustic/CausticGrammar/src/caustic/grammar/test/sub.pg:1:7:"Rule:   **> ITEM+[SEP]" => Unexisting module "ITEM_1_base" in reference "ITEM_1_base.sub.SEP"

[igordejanovic]

Looks like a bug. Thanks for the detailed report.

[Tiger-Tom]

Let me know if you need any more details. This is a great library and I'm happy to help in whatever way I can.

[igordejanovic]

Would you be able to provide a failing test for this?