antlr / grammars-v4

Grammars written for ANTLR v4; expectation that the grammars are free of actions.
MIT License
9.87k stars 3.66k forks source link

golang parse python3 failed #3995

Open kayguo opened 3 months ago

kayguo commented 3 months ago

https://github.com/antlr/grammars-v4/blob/master/python/python3/Go/python3_lexer_base.go

parse fail : we will lost the first token

we can fix it after "next := l.BaseLexer.NextToken()" : add l.tokens ,like this: if len(l.tokens) != 0 && l.tokens[len(l.tokens)-1] != next { l.tokens = append(l.tokens, next) }

kaby76 commented 3 months ago

The suggested change would cause a significant deviation from all the other sources, creating a maintenance problem.

The main problems, which are multiple, are described here. https://github.com/antlr/antlr4/issues/4342 OO is not available in Go. Fine. But, one can work around this by implementing it explicitly using structs and pointers to base class objects. I implemented things like this over 30 years ago in C. I know it can be done. The Go target runtime tries to do the same, but it does not implement it correctly.