goccy / go-yaml

YAML support for the Go language
MIT License
1.12k stars 129 forks source link

fix: Correct token.Tokenize double quoted strings with escape sequences handling #457

Closed nieomylnieja closed 1 month ago

nieomylnieja commented 2 months ago

Before submitting your PR, please confirm the following.

I've rewritten lexer_test.go TestTokenize test function to actually test something instead of dumping the tokens :)

I've generated these test cases by writing the following code in the TestTokenize:

f, _ := os.Create("lexer_test_cases.go")
defer f.Close()
for _, test := range tests {
    tokens := lexer.Tokenize(test.YAML)

    builder := strings.Builder{}
    for _, tk := range tokens {
        builder.WriteString(fmt.Sprintf(`{
            Type:          token.%sType,
            CharacterType: token.CharacterType%s,
            Indicator:     token.%sIndicator,
            Value:         %q,
            Origin:        %q,
        },
    `, tk.Type, tk.CharacterType, tk.Indicator, tk.Value, tk.Origin))
    }
    fmt.Fprintf(f, `{
        YAML: `+"`"+`%s`+"`"+`,
        Tokens: token.Tokens{
            %s
        },
    },
    `, test.YAML, builder.String())
}

I then just copied the generate code fro lexer_test_cases.go into tokens struct list assignment.

codecov-commenter commented 2 months ago

:warning: Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 78.70%. Comparing base (b2a8cc6) to head (820faf4).

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #457 +/- ## ========================================== + Coverage 78.65% 78.70% +0.05% ========================================== Files 13 13 Lines 3983 3983 ========================================== + Hits 3133 3135 +2 + Misses 590 589 -1 + Partials 260 259 -1 ```