syntax: `$var#...` is incorrectly parsed as a comment

[emilazy]

Example using shfmt:

emily@yuyuko ~> cat test.bash
foo=abc
bar=def
echo foo#bar
echo foo#$bar
echo $foo#bar
echo $foo#$bar
emily@yuyuko ~> bash test.bash
foo#bar
foo#def
abc#bar
abc#def
emily@yuyuko ~> shfmt test.bash
foo=abc
bar=def
echo foo#bar
echo foo#$bar
echo $foo #bar
echo $foo #$bar
emily@yuyuko ~> shfmt test.bash | bash
foo#bar
foo#def
abc
abc

None of these contain comments, but the latter two examples get turned into comments (and this actually came up in practice; see https://github.com/bouk/babelfish/issues/23). I'm not sure if it's just variable substitutions or if there are other circumstances where this incorrect parse happens.

[bouk]

Seems like adding a spaced check here is all that's needed: https://github.com/mvdan/sh/blob/9e323b8552f73daa5fd63f7dff0dfbc3f7bc3af8/syntax/lexer.go#L272

[mvdan]

Nice find - genuinely surprised that noone spotted this bug before. @bouk has the right idea, but unfortunately the fix isn't that easy. I'm investigating.

[mvdan]

I spent quite a bit of time overthinking different ways to fix this, but I landed on something relatively simple: a small variation on @bouk's idea. It's not a complete fix, as two other cases still don't parse the way they should, but the cases you showed in this issue are all fixed.

[emilazy]

Thanks for this! Would it be good to open an issue for the remaining cases? I haven't seen nix shell $(calculate-flake-ref)#... type constructions appear in a script yet, but I think more out of luck than anything else.

[mvdan]

I left TODOs in the code as a personal reminder, but please feel free to open another issue if you think you'll run into it soon. Particularly if you already have any real example :)

[emilazy]

I'll wait to run into it :)

[bouk]

@mvdan if you cut a release I'll update babelfish!

[mvdan]

done :)