Lexer.Generator is slow in generating of a token with many definitions

[ForNeVeR]

I have defined a simple lexer:

public enum TokenType
{
    [Error] Error,
    [End] End,

    [Token("auto")]
    [Token("break")]
    [Token("case")]
    [Token("char")]
    [Token("const")]
    [Token("continue")]
    [Token("default")]
    [Token("do")]
    [Token("double")]
    [Token("else")]
    [Token("enum")]
    [Token("extern")]
    [Token("float")]
    [Token("for")]
    [Token("goto")]
    [Token("if")]
    [Token("inline")]
    [Token("int")]
    [Token("long")]
    [Token("register")]
    [Token("restrict")]
    [Token("return")]
    [Token("short")]
    [Token("signed")]
    [Token("sizeof")]
    [Token("static")]
    [Token("struct")]
    [Token("switch")]
    [Token("typedef")]
    [Token("union")]
    [Token("unsigned")]
    [Token("void")]
    [Token("volatile")]
    [Token("while")]
    [Token("_Alignas")]
    [Token("_Alignof")]
    [Token("_Atomic")]
    [Token("_Bool")]
    [Token("_Complex")]
    [Token("_Generic")]
    [Token("_Imaginary")]
    [Token("_Noreturn")]
    [Token("_Static_assert")]
    [Token("_Thread_local")]
    Keyword
}

[Lexer(typeof(TokenType))]
public partial class CLexer { }

And this lexer gets generated for a minute every time I recompile the library.

Which libraries does it affect? Yoakke.Lexer.Generator.

Expected behavior I think that such a keyword list shouldn't take that long time to generate.

Environment (please complete the following information):

• OS: Windows 11
• .NET version: .NET 6
• Version: <PackageReference Include="Yoakke.Lexer.Generator" Version="2021.12.18-2.37.17-nightly" />

[LPeter1997]

Thanks for the report! As it turns out, the DFA minimization takes quite bit because of the big state space. There's some low-hanging fruit I'll try to pick now. I'll release a fix as soon as I feel like I've made a decent improvement.

[LPeter1997]

@ForNeVeR I think the issue is fixed in the latest nightly (2021.12.18-18.21.47-nightly). Please provide feedback, if the issue is fixed on your part. Feel free to reopen the issue, if it persists.

[ForNeVeR]

@LPeter1997, first of all, you react to issues and work amazingly fast.

Second: my builds of the same lexer are now amazingly fast, too. From a minute down to 2 seconds.

Thanks a lot for your instant help!