Making gocc practical for normal sized projects

goccmack / gocc

Parser / Scanner Generator

Other

618 stars 48 forks source link

Making gocc practical for normal sized projects #132

Open suntong opened 2 years ago

suntong commented 2 years ago

My gocc has been running for over a day now at 100% CPU:

First time seeing it, hopefully there is no endless loop caused by my grammar.

suntong commented 2 years ago

Ah, it finally stopped:

real    2186m58.446s
user    2412m45.462s
sys     5m40.899s

2187/60/24 = 1.51875 -- It ran for one and half days.

suntong commented 2 years ago

3+ days now.

awalterschulze commented 2 years ago

Can you share the grammar? This is really weird.

suntong commented 2 years ago

Oh, thanks for the reply. Here it is:

https://github.com/suntong/lang/blob/master/lang/Go/src/parsers/gocc/grm6-sqldef/sqldef.bnf

It's bit long (still unfinished), but still less than 1000 lines.
In prospect, https://github.com/antlr/grammars-v4/blob/master/sql/tsql/TSqlParser.g4 alone has nearly 4700 lines, I.e., nearly 5 times longer, and if plus the Lexer it’ll be 6~7 times long, yet that one I believe compiles in reasonable time. Anything wrong with my syntax/usage? I guess I did put some heavy burden on the lexer side...

BTW, another day has gone by and it is still running.

suntong commented 2 years ago

I have to stop it now, as it is taking too much of my memory and I cannot perform my normal daily work now,

It's taking almost half of my 32G memory, and it has been running straight for about eight and half days.

real    12197m20.267s
user    15062m32.699s
sys     32m45.269s

12197/60/24 = 8.47014

I wish I can see when it can finish, but I gave it up now. Somebody try it please to run gocc on https://github.com/suntong/lang/blob/master/lang/Go/src/parsers/gocc/grm6-sqldef/sqldef.bnf.

Thanks

awalterschulze commented 2 years ago

This is really weird @goccmack would you like to take a look?

goccmack commented 2 years ago

@awalterschulze @suntong

It appears that you have created loops in your grammar. I commented out all the productions that were causing unused warnings and now gocc completes:

time gocc sqldef.bnf gocc sqldef.bnf 9.21s user 0.14s system 135% cpu 6.905 total

This is still very long so I suspect that there are problems in in your lexical spec too.

At some stage gocc should catch this type of error but I don't have the time to implement it now.

suntong commented 2 years ago

Thanks a lot @goccmack, so you commented out only the unused warnings, and now the problem is gone. OK, I'll try to duplicate that at my end, just currently I've lost that building environment that I had before, and am building a brand new machine now. so I'll try to get back to you in a few days... thx again.