tdewolff / parse

Go parsers for web formats
MIT License
413 stars 65 forks source link

JSON is parsed as valid Javascript #67

Closed Jacob-Roberts closed 3 years ago

Jacob-Roberts commented 3 years ago

When using this library with the following code:

l := js.NewLexer(parse.NewInputBytes(data))
for {
    tt, _ := l.Next()
    if tt == js.ErrorToken {
        if l.Err() != io.EOF {
            return false, "The Javascript file is not valid", l.Err()
        } else {
            return true, "", nil
        }
    }
}

and the following input

{
    "elementId": "mJn9mpMQwIYyBe",
}

No error is detected. This returns true.

tdewolff commented 3 years ago

Hey @Jacob-Roberts , that is correct! The lexer does not detect semantic errors (and it shouldn't!) but just makes sure that the input can be converted into valid JS tokens (such as keywords, strings, number literals, and interpunction/operators). In this case the source contains all valid tokens. If you want to find semantic errors, you should run the parser instead (js.NewParser). This should return the following:

ERROR: cannot minify in.js: unexpected : in expression on line 2 and column 16
    2:     "elementId": "mJn9mpMQwIYyBe",
                      ^

Let me know if that works for you or whether you have additional questions! ;-)

Jacob-Roberts commented 3 years ago

Thank you!

Jacob-Roberts commented 3 years ago

Maybe as an improvement point, you could point out in the Readme for the JS page that the parser is available. It seems from the README that the Lexer is the only option

tdewolff commented 3 years ago

Good idea, I've added a short description, hopefully that covers it.