Collision between token ID and token lexeme

[mewmew]

Contents of x.bnf:

foo : 'b' 'a' 'r' ;

Start : foo "foo" ;

$ gocc x.bnf
panic: Production foo already exists

goroutine 1 [running]:
panic(0x5a4220, 0xc820011d60)
    /home/u/go/src/runtime/panic.go:500 +0x18a
github.com/goccmack/gocc/ast.(*LexProdMap).Add(0xc820011c30, 0xc8200a1c88, 0x1, 0x1)
    /home/u/goget/src/github.com/goccmack/gocc/ast/lexprodmap.go:67 +0x2d0
github.com/goccmack/gocc/ast.(*LexPart).UpdateStringLitTokens(0xc8200b0000, 0xc820086400, 0x1, 0x10)
    /home/u/goget/src/github.com/goccmack/gocc/ast/lexpart.go:128 +0xec
main.main()
    /home/u/goget/src/github.com/goccmack/gocc/main.go:97 +0x5af

Gocc revision c7163b52d2f23f5cae8af3b9fc81d0ea0fe8cd42.

[mewmew]

This error was encountered while implementing a Gocc BNF for LLVM IR, the relevant extract of which is presented below.

// ## Identifiers

_name        : _letter { _letter | _decimal_digit } ;
_quoted_name : _quoted_string ;
_id          : _decimals ;

// ### Global Identifiers

global
    : _global_name
    | _global_id
;

_global_name : '@' ( _name | _quoted_name ) ;
_global_id   : '@' _id ;

// # Declarations

GlobalVarDecl : global "=" "global" Type Value ;

The global token ID collides with the token lexeme "global".

I seem to have a memory of you guys mentioning that this was fixed in Gocc 3. If so, how did you fix it?

[awalterschulze]

@goccmack I think you should answer this one.

[mewmew]

@goccmack and @awalterschulze Have you gained any further insight into the cause of this issue?

It seems to still be present in the latest version of gocc (rev e7acd082238fc5a6f8ae224308f4dbf624c55cde).

The example below uses the same grammar for x.bnf, as reported in https://github.com/goccmack/gocc/issues/20#issue-148975168

$ gocc x.bnf 
panic: Production foo already exists

goroutine 1 [running]:
panic(0x5ed200, 0xc4200b6060)
    /home/u/go/src/runtime/panic.go:500 +0x1a1
github.com/goccmack/gocc/ast.(*LexProdMap).Add(0xc420011f30, 0xc42009dc70, 0x1, 0x1)
    /home/u/goget/src/github.com/goccmack/gocc/ast/lexprodmap.go:67 +0x2d1
github.com/goccmack/gocc/ast.(*LexPart).UpdateStringLitTokens(0xc4200ac3f0, 0xc420082400, 0x1, 0x10)
    /home/u/goget/src/github.com/goccmack/gocc/ast/lexpart.go:128 +0xee
main.main()
    /home/u/goget/src/github.com/goccmack/gocc/main.go:97 +0x5e0

[awalterschulze]

I have not had any time to look at this. I was hoping @goccmack would take a look, but I think he is really busy at the moment.

Would you mind taking a look @mewmew ?

[mewmew]

Would you mind taking a look @mewmew ?

Sure. The only thing I was curious about before diving into the code is whether this ever was fixed in Gocc 3, and if it would then make sense to look in that code base. And I guess, this is something @goccmack would know. Any ideas?

[goccmack]

The reason for this error is that the gocc2 symbol table is not sophisticated enough to distinguish between an explicitly declared symbol (e.g. foo : ...) and and implicitly declared symbol (e.g.: "foo").

It looks like gocc3 RC3 still had the same problem.

[mewmew]

Oh, I see. Thank you Marius for providing some insight into this. I may try to tackle the issue within the months to come : )

Cheerful regards from a snow-filled Sweden.