textmate / shellscript.tmbundle

TextMate support for ShellScript
43 stars 29 forks source link

[bash] commands after HEREDOC delimiter are highlighted like the HEREDOC #35

Open aeschli opened 7 years ago

aeschli commented 7 years ago

From @dexwerx on January 5, 2017 20:59

Steps to Reproduce:

  1. Create a Shell Script
  2. Paste
    # grab field types from ORACLE
    sqlplus -s user/pass << EOD | sed '/^$/d' | awk 'NR>2{print}' > temp.txt
    SET PAGESIZE 0;
    SET FEEDBACK OFF;
    DESCRIBE $1;
    EOD

everything after the end of heredoc token EOD is highlighted the same as the rest of the heredoc.

Copied from original issue: Microsoft/vscode#18192

infininight commented 7 years ago

Not seeing this issue in TextMate, the regex here looked reasonably straightforward so nothing jumps out as being the issue. My first suspicion is a different in regex engine syntax but I know basically nothing about how vsCode is implemented language-wise.

mkhl commented 7 years ago

I believe that there are two issues here:

  1. The code as pasted doesn’t highlight anything as a heredoc. The grammar doesn’t allow whitespace in between << and the heredoc token, although it should. I believe we’ve recently fixed this for generic redirections but not for heredocs.

  2. With heredoc highlighting fixed (or when removing the space before EOD), the remainder of the line starting the heredoc is also treated as a heredoc, although it shouldn’t. Fixing this is more involved and I’m still thinking about how best to do this.

    A short sketch of what I’m thinking: Our current rules describe a heredoc as beginning and ending at the token. Instead, the beginning token could open a scope with two rules:

    1. begin: lookbehind at the heredoc operator and token, end: newline, include: $self and possibly a rule matching backslash-escaped newlines (to prevent those from ending this scope)
    2. begin: lookbehind at newline (or lookahead at anything), end: lookahead at heredoc token The second rule would provide the actual heredoc scope. I don’t think child rules can access their parents’ matches, so the actual heredoc token wouldn’t be available there. We could try to treat anything like possible heredoc tokens, relying on the fact that if the parent’s end fails the second rule’s begin would match.

    This problem also seems to be present in the Ruby bundle.

sorbits commented 7 years ago

There is also the possibility of matching the (start) heredoc token and then capture the rest of the line, and for the captures part of the rule, include $self, this would avoid the nested scope, but it would not be possible to handle escaped newlines after the heredoc token.

dexwerx commented 6 years ago

annual bump!

IvanRoman commented 6 years ago

I was wondering if there is any update on this. I can see this issue on Ubuntu 18.04.1 with VS Code v. 1.27.2

NorseGaud commented 5 years ago

2019 bump!!! This is extremely annoying.

On Mojave.

Mist-Hunter commented 4 years ago

Same problem. 2020 bump, just started to get on the vscode bus, but I use heredoc's allot..

Mist-Hunter commented 4 years ago

Same problem. 2020 bump, just started to get on the vscode bus, but I use heredoc's allot..

FYI, quoting EOT seems to fix this. Example.

No quotes, not working image

Quotes (double below, but single also works) working. Using double due presence of variables image

Mist-Hunter commented 4 years ago

Update to myself. The quotes didn't work, they actually broke variable expansion. Randomly though I discovered that you need a space after EOT. Bash doesn't care, but VSCode does. To be fair, many pages do show the spaces: https://linuxize.com/post/bash-heredoc/ ,so even though they aren't needed by bash, I'm not sure this is a bug.

This may be well known by others, but I'm guessing this bug wouldn't be here if everyone knew.

cat < makes VS Code sad

cat < gets highlighted properly and works!

jannek-aalto commented 4 years ago

Still broken in vscode 1.49.2 for ksh space-trimming heredoc syntax '<<#word'.

FOO=$(cat <<#EOT
  foo
  bar
EOT
)