Visitor Pattern in Golang

[kaby76]

Try implementing a dispatch routine. E.g.,

func (v *Visitor) Visit(tree antlr.ParseTree) any {
    switch val := tree.(type) {
    case *parser.ProgContext:
        return v.VisitProg(val)
    case *parser.DeclContext:
        return v.VisitDecl(val)
    case *parser.ExprContext:
        return v.VisitExpr(val)
    }
    return 0
}

[kaby76]

There is already a Visit method in the base class which I use and should be used.

Of course. The Antlr Visitor code doesn't work. Likely for Listeners as well.

https://github.com/antlr/antlr4/issues/2504 https://github.com/antlr/antlr4/issues/3265 https://github.com/antlr/antlr4/discussions/3812 https://github.com/antlr/antlr4/issues/1843

But, you asked: "Anyone [have] any idea how the visitor pattern should work in Go?". I am just saying, that if you are trying to get something to work now, then you might want to give that a try.

[kaby76]

I think an end-to-end example of a Visitor is in order. The doc only shows a Listener. Both are needed.

I could not get Antlr Visitors to work unless I do the following:

• Define Visitor as you describe:

  type Visitor struct {
      parser.BaseExprVisitor
  }

• Define Visit() as you describe:

  func (v *Visitor) Visit(tree antlr.ParseTree) any {
      return tree.Accept(v)
  }

• Define each and every Visit...() function:

  func (v *Visitor) VisitProg(ctx *parser.ProgContext) any {
      fmt.Println("VisitProg")
      return v.VisitChildren(ctx)
  }
  
  func (v *Visitor) VisitDecl(ctx *parser.DeclContext) any {
      fmt.Println("VisitDecl")
      return v.VisitChildren(ctx)
  }
  
  func (v *Visitor) VisitExpr(ctx *parser.ExprContext) any {
      fmt.Println("VisitExpr")
      return v.VisitChildren(ctx)
  }

  In other words, the generated Visit...() functions do not work:

  func (v *BaseExprVisitor) VisitProg(ctx *ProgContext) interface{} {
      return v.VisitChildren(ctx)
  }
  
  func (v *BaseExprVisitor) VisitDecl(ctx *DeclContext) interface{} {
      return v.VisitChildren(ctx)
  }
  
  func (v *BaseExprVisitor) VisitExpr(ctx *ExprContext) interface{} {
      return v.VisitChildren(ctx)
  }

• Define VisitChildren(), because the default implementation is "return nil".

  func (v *Visitor) VisitChildren(tree antlr.RuleNode) any {
      n := tree.GetChildCount()
      for i := 0; i < n; i++ {
          c := tree.GetChild(i)
          val := c.(antlr.ParseTree)
          _ = v.Visit(val)
      }
      return 0
  }

  Note, this implementation does not "sum" the results of visiting the children, nor does it "short-circuit" the tree walking.

Also, in the example in the doc for the listener, the struct type declares the base as pointer. https://github.com/antlr/antlr4/blob/a27cf84839c4f794e43b83b9b922d9342367fe45/doc/go-target.md?plain=1#L244 If I try to do the same for a Visitor, the tree walker crashes because the VisitErrorNode() and VisitTerminal() funcs also need to be defined:

func (v *Visitor) VisitErrorNode(_ antlr.ErrorNode) interface{}   { return nil }
func (v *Visitor) VisitTerminal(_ antlr.TerminalNode) interface{} { return nil }

VisitProg() must still be defined.

[jimidle]

I use both. I have not had time to look at what is happening here, but you don’t need to implement all funcs. Just the ones you need.

I do wish to make some improvements to the way variables are accessed. But I have not had time to do so. It does all work though. If you are new to go, then it might not be obvious. I should create some example projects to help.

Jim

On Mon, Aug 28, 2023 at 09:09 Ken Domino @.***> wrote:

I think an end-to-end example is in order. I could not get Antlr Visitors to work unless I do the following:

• Define Visitor as you describe:

  type Visitor struct { parser.BaseExprVisitor }

• Define Visit() as you describe:

  func (v *Visitor) Visit(tree antlr.ParseTree) any { return tree.Accept(v) }

• Define each and every Visit...() function:

  func (v Visitor) VisitProg(ctx parser.ProgContext) any { fmt.Println("VisitProg") return v.VisitChildren(ctx) }

  func (v Visitor) VisitDecl(ctx parser.DeclContext) any { fmt.Println("VisitDecl") return v.VisitChildren(ctx) }

  func (v Visitor) VisitExpr(ctx parser.ExprContext) any { fmt.Println("VisitExpr") return v.VisitChildren(ctx) }

  In other words, the generated Visit...() functions do not work:

  func (v BaseExprVisitor) VisitProg(ctx ProgContext) interface{} { return v.VisitChildren(ctx) }

  func (v BaseExprVisitor) VisitDecl(ctx DeclContext) interface{} { return v.VisitChildren(ctx) }

  func (v BaseExprVisitor) VisitExpr(ctx ExprContext) interface{} { return v.VisitChildren(ctx) }

• Define VisitChildren(), because the default implementation is "return nil".

  func (v *Visitor) VisitChildren(tree antlr.RuleNode) any { n := tree.GetChildCount() for i := 0; i < n; i++ { c := tree.GetChild(i) val := c.(antlr.ParseTree) _ = v.Visit(val) } return 0 }

  Note, this implementation does not "sum" the results of visiting the children, nor does it "short-circuit" the tree walking.

— Reply to this email directly, view it on GitHub https://github.com/antlr/antlr4/issues/4395#issuecomment-1694846647, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJ7TMFMT5QI2UF2N3SPOGLXXPVTRANCNFSM6AAAAAA37NBKR4 . You are receiving this because you were mentioned.Message ID: @.***>

[kaby76]

The code attached implements a Visitor for the Java and Go targets. I have one for CSharp, but it behaves like the Java target, so I haven't included it. The applications are generated for Windows Msys2.

In this example, I define all Visit...() methods/func except ProgVisit().

For Java, the tree walker visits the entire tree, which you can see by the output for DeclVisit() and ExprVisit().

For Go, the visitor does not appear to visit the Expr and Decl nodes because "func ProgVisit()" is commented out. If "func ProgVisit()" is uncommented, I get the output from the ExprVisit() and DeclVisit() funcs. I haven't determined whether it calls the Antlr tool generated func's, but it certainly doesn't call the ones that I define.

08/28-02:29:36 ~/issues/antlr-4395/x/Generated-Java
$ ls
build.sh  ErrorListener.java  ExprBaseVisitor.java  makefile                        perf.sh  Test.java  test1.txt
clean.sh  Expr.g4             ExprVisitor.java      MyDiagnosticErrorListener.java  run.sh   test.sh    Visitor.java
08/28-02:29:37 ~/issues/antlr-4395/x/Generated-Java
$ make
bash build.sh
08/28-02:29:45 ~/issues/antlr-4395/x/Generated-Java
$ bash run.sh test1.txt
visitDecl
visitDecl
visitExpr
visitExpr
visitExpr
visitExpr
visitExpr
visitExpr
visitDecl
Java 0 test1.txt success 0.011
Total Time: 0.115
08/28-02:30:16 ~/issues/antlr-4395/x/Generated-Java
$ cd ..
08/28-02:30:19 ~/issues/antlr-4395/x
$ cd Generated-Go/
08/28-02:30:23 ~/issues/antlr-4395/x/Generated-Go
$ ls
build.sh  clean.sh  go.mod  go.sum  makefile  parser/  perf.sh  run.sh  Test.go  test.sh  test1.txt
08/28-02:30:26 ~/issues/antlr-4395/x/Generated-Go
$ make
bash build.sh
08/28-02:30:32 ~/issues/antlr-4395/x/Generated-Go
$ bash run.sh test1.txt
Go 0 test1.txt success 0.000
Total Time: 0.001 s
08/28-02:30:40 ~/issues/antlr-4395/x/Generated-Go
$

Here are the two applications in .zip files.

Generated-Java.zip Generated-Go.zip

Again, just looking through the Antlr Go runtime, there's no func that defines VisitChildren() that actually traverses all children of a tree node. The only VisitChildren() implementation is the stub implementation in tree.go.

08/28-02:54:29 /c/Users/Kenne/Documents/GitHub/antlr4-dev/runtime/Go/antlr/v4
$ grep VisitChildren *.go
parser_rule_context.go: return visitor.VisitChildren(prc)
tree.go:        VisitChildren(node RuleNode) interface{}
tree.go:func (v *BaseParseTreeVisitor) VisitChildren(_ RuleNode) interface{}     { return nil }
08/28-02:54:38 /c/Users/Kenne/Documents/GitHub/antlr4-dev/runtime/Go/antlr/v4
$ pushd
~/issues/antlr-4395/x/Generated-Go /c/Users/Kenne/Documents/GitHub/antlr4-dev/runtime/Go/antlr/v4
08/28-02:57:48 ~/issues/antlr-4395/x/Generated-Go
$ grep VisitChildren *.go
        return v.VisitChildren(ctx)
        return v.VisitChildren(ctx)
        return v.VisitChildren(ctx)
func (v *Visitor) VisitChildren(tree antlr.RuleNode) any {
08/28-02:57:59 ~/issues/antlr-4395/x/Generated-Go
$ grep VisitChildren parser/*.go
parser/expr_base_visitor.go:    return v.VisitChildren(ctx)
parser/expr_base_visitor.go:    return v.VisitChildren(ctx)
parser/expr_base_visitor.go:    return v.VisitChildren(ctx)
parser/expr_parser.go:          return t.VisitChildren(s)
parser/expr_parser.go:          return t.VisitChildren(s)
parser/expr_parser.go:          return t.VisitChildren(s)
08/28-02:58:15 ~/issues/antlr-4395/x/Generated-Go
$

Every other runtime has an implementation somewhere to loop through all children of a parse tree node. I just don't see how this could have ever worked for Go--unless one implements a VisitChildren(). as I have done

• CSharp. https://github.com/antlr/antlr4/blob/a27cf84839c4f794e43b83b9b922d9342367fe45/runtime/CSharp/src/Tree/AbstractParseTreeVisitor.cs#L44-L59
• Cpp. https://github.com/antlr/antlr4/blob/a27cf84839c4f794e43b83b9b922d9342367fe45/runtime/Cpp/runtime/src/tree/AbstractParseTreeVisitor.h#L35-L48
• Dart. https://github.com/antlr/antlr4/blob/a27cf84839c4f794e43b83b9b922d9342367fe45/runtime/Dart/lib/src/tree/src/tree.dart#L117-L131
• Java. https://github.com/antlr/antlr4/blob/a27cf84839c4f794e43b83b9b922d9342367fe45/runtime/Java/src/org/antlr/v4/runtime/tree/AbstractParseTreeVisitor.java#L37-L51
• JavaScript/TypeScript. https://github.com/antlr/antlr4/blob/a27cf84839c4f794e43b83b9b922d9342367fe45/runtime/JavaScript/src/antlr4/tree/ParseTreeVisitor.js#L6-L22
• Python3. https://github.com/antlr/antlr4/blob/a27cf84839c4f794e43b83b9b922d9342367fe45/runtime/Python3/src/antlr4/tree/Tree.py#L36-L47
• Swift. https://github.com/antlr/antlr4/blob/a27cf84839c4f794e43b83b9b922d9342367fe45/runtime/Swift/Sources/Antlr4/tree/AbstractParseTreeVisitor.swift#L33-L48
• PHP (*no 4.13.0 release; does not work with 4.13). https://github.com/antlr/antlr-php-runtime/blob/1ebacfd8a9761dc14509c305887d43899f1e18c5/src/Tree/AbstractParseTreeVisitor.php#L39-L59

[kaby76]

Nice. But this is not the visitor pattern:

func (v *Visitor) Visit(tree antlr.ParseTree) any {
    return tree.Accept(v)
}

The method from the base class should be used.

That function is essentially the code that you gave above!

func (v *Visitor) Visit(tree antlr.ParseTree) any {
    return tree.accept(v)
}

But, besides, the implementation I made is equivalent to the "base class" implementation except for the function receiver. https://github.com/antlr/antlr4/blob/a27cf84839c4f794e43b83b9b922d9342367fe45/runtime/Go/antlr/v4/tree.go#L61

Doing what you suggest--commenting out the Visit() implementation in Test.go that I made--does not work either.

It's moot anyway because the overarching fact remains: VisitChildren() is unimplemented, yet the Antlr tool generates code with VisitChildren() function calls. The Visitor code can only work with all the func's implemented as I mentioned above.

[kaby76]

It's moot anyway because the overarching fact remains: VisitChildren() is unimplemented, yet the Antlr tool generates code with VisitChildren() function calls. The Visitor code can only work with all the func's implemented as #4395 (comment).

It's not moot. It's a bug!

I really don't know what you are referring to. What is "it" in "It's a bug!"?? So as to be clear, I've checked in all the generated driver code here, https://github.com/kaby76/antlr-4395, so you can use Github Permalinks to address what "it" is. Thanks.