Dottie doesn't handle multiline variables

[intentionally-left-nil]

Create a .env file with:

multi="first
second"

Expected: Dottie parses multi as a variable containing a two-line value. Actual: error Error: unexpected token at line 2: Illegal(first) - parseRowStatement 1

[intentionally-left-nil]

I took a quick look and this is going to be a non-trivial fix. The logic error begins with scanQuotedValue

where it returns an error if a newline is found

if isEOF(s.rune) || isNewLine(s.rune) {
    tType = token.Illegal

    break
}

However, it's not as simple as removing the isNewLine check, because the Token struct makes assumptions that it is only applied over a single line. So E.g. the token length is wrong.

type Token struct {
    Type       Type
    Literal    string
    Offset     int
    Length     int
    LineNumber uint
    Commented  bool
    Quote      Quote
    Annotation *Annotation
}

So this is going to require some more heavy redesigning of your lexer to account for the fact that tokens can span multiple lines.

The other option I kind of considered would be a pre-processor pass to convert "hello world"

to "hello\nworld" (with the literal \n in the string) but that has its own complexities (e.g. first you have to keep track of escaped sequences, find the proper closing quote etc.

and it feels like a better solution is just to fix the real logic of your scanner

[intentionally-left-nil]

P.P.S -

While I get that it's fun to write your own lexer/parser for your own code-> Have you considered any of the standardized formats over the last 50 years and reusing an existing tool? Especially as you're having other projects (pixelfed) use this library.

For example, for golang, there's goyacc https://www.dolthub.com/blog/2023-07-28-goyacc-parser-tips-tricks/ and you could replace most of this code with just your formal grammar of the .env file

[jippi]

There really aren't any consistent .env grammar out there, so many tools treat them differently, and my parser handles a bunch of those weird mixes rather well.

For Pixelfed; its quite easy to disable dottie syntax checking for cases where it does not work - it has however cause a lot of misconfigurations, so was left on by default for those use-cases :)

[intentionally-left-nil]

Ah nice, I missed that environment option :) Yay for chicken bits

[jippi]

intentionally designed it so all the nice and convenient validation and helpers can be disabled for folks who don't need it; or is hit by bugs (or lacks of features) and easily circumvent that and move on :)

Golden Path with easy off-ramps

[jippi]

I also fixed the issue in https://github.com/jippi/dottie/pull/41 :)