Creating a parser instance inside a BNF file

[shivansh]

I'm trying to create a parser instance inside a bnf file. Interestingly, when I try to do so the following error is encountered -

tmp/parser/productionstable.go:320: initialization loop:
    /home/zeebu/sourceCodes/personalProjects/goProjects/src/gogo/tmp/parser/productionstable.go:320 productionsTable refers to
    /home/zeebu/sourceCodes/personalProjects/goProjects/src/gogo/tmp/parser/productionstable.go:113 glob.func3.1 refers to
    /home/zeebu/sourceCodes/personalProjects/goProjects/src/gogo/tmp/parser/parser.go:176 (*Parser).Parse refers to
    /home/zeebu/sourceCodes/personalProjects/goProjects/src/gogo/tmp/parser/productionstable.go:320 productionsTable

I think one way to escape the initialization loop is to initialize productionsTable inside init() function, but then none of the reduce functions inside productionsTable get called (not sure why, and also it will be an extra effort to make changes in productionstable.go everytime it is generated from the BNF).

Is there any other way to achieve the same ?

PS. The issue is not specifically related to gocc, but any help will be greatly appreciated.

[mewmew]

Not sure if your very goal is to keep all Go code in the production action (i.e. inside << and >>). Otherwise, you could factor out the code into a package and invoke it as a function. This approach is the one I personally use when the production action becomes more complex than a simple return statement.

See for instance:

https://github.com/llir/llvm/blob/master/asm/internal/ll.bnf#L785 https://github.com/llir/llvm/blob/master/asm/internal/astx/astx.go#L1509

[mewmew]

P.S. to make the package Go-getable, you may wish to replace your import paths with their fully qualified paths (i.e. to include the github.com/shivansh prefix).

https://github.com/shivansh/gogo/blob/imports/src/imports.bnf#L46

-"gogo/tmp/token"
-"gogo/tmp/lexer"
+"github.com/shivansh/gogo/tmp/token"
+"github.com/shivansh/gogo/tmp/lexer"

Note, I couldn't find the tmp directory in your repo, so perhaps the import paths would have to be updated further, but you see the general idea.

[shivansh]

@mewmew Thanks a lot. I tried a somewhat similar approach earlier, but due to our project structure we encountered an import loop. Although I guess it will be possible to solve it with some modifications.

tmp is a locally generated directory ([1] [2]) which contains all the files generated by gocc. We're planning to remove the generated files from gitignore and add them to the source soon (our bnf is about to get stable).

PS. Will close the issue once the entire problem is solved (just in case if there are some more hurdles on the way).

[mewmew]

I tried a somewhat similar approach earlier, but due to our project structure we encountered an import loop.

For this reason, I've been using an astx package specifically for BNF production rules, and separate ast and ir packages, so that no import cycles or unnecessary dependencies are introduced.

[shivansh]

@mewmew So I went ahead and following yours and @awalterschulze's instructions refactored the functions for SDT rules into a separate ast package.

Coming to the problem of import cycle, here is how the function for handling import declarations looks as of now. In doing so I've realized that it might not be possible for me to import and use the parser instance without introducing an import cycle due to my project structure.

gogo/src/parser -> gogo/tmp/parser -> gogo/src/ast  (for making IR generation work)
gogo/src/ast -> gogo/tmp/parser                     (for making imports work)

lhs -> rhs implies lhs imports rhs.
gogo/src/parser is gogo's parser package.
gogo/tmp/parser is gocc's parser package (auto generated on compile).

I've tried many variants of the above import graph, but all of them had a cycle. The only way I was able to escape the import cycle was by separating out the NewParser() function (available in gocc's parser package) into a new package, but that will not be feasible :sweat_smile:

I might've hit a dead end here.

[awalterschulze]

The generated parser has to import the ast package. There is no getting around that. So the ast package cannot import the generated parser package.

So like you said this is causing your problem:

func NewImportDecl(decl, declList Attrib) (Node, error) {
    imports := decl.(Node).Code
    imports = append(imports, declList.(Node).Code...)
    fmt.Println(imports)
    for _, v := range imports {
        s := lexer.NewLexer(content)
        p := parser.NewParser()
        _, err = p.Parse(s)
        if err != nil {
            log.Fatal(err)
        }
    }
    return Node{}, nil
}

I would try to get rid of the need for an ast function to parse?

Maybe parsing is one step and "linking" the imports together is another? That way you parse each file separately and then have a second step which links them together.

But maybe I am misinterpreting the need for parsing in NewImportDecl ?