Docker heredoc support

[jin-ahn]

LSP doesn't support heredoc syntax properly. https://www.docker.com/blog/introduction-to-heredocs-in-dockerfiles/

[rcjsuen]

@jin-ahn Hi, thank you for opening an issue. Support for heredocs has been present since August 2021 and we have continued to build on top of that work in the last year.

Could you please provide a screenshot of the errors you are seeing. What LSP client are you using?

[jin-ahn]

If you see my screen shots once I have a heredoc portion in my Dockerfile the syntax highlighting below doesn't work, but more importantly none of the lsp works.

Also here is my LSP client

[rcjsuen]

@jin-ahn Have you tried to build your Dockerfile? It looks malformed to me. If you look at the examples in your blog post they do not put spaces in front of the EOF line.

# syntax=docker/dockerfile:experimental
FROM alpine
RUN <<EOF
  echo "hello world"
  EOF

RUN echo "hello world 2"

$ DOCKER_BUILDKIT=1 docker build .
[+] Building 0.0s (0/1)                                                                                                                                                                                                                 [+] Building 0.1s (2/3)                                                                                                                                                                                                                 

 => [internal] load build definition from Dockerfile                                                                                                                                                                                        0.0s
 => => transferring dockerfile: 31B                                                                                                                                                                                                         0.0s
[+] Building 0.3s (3/3)                                                                                                                                                                                                                          
 => [internal] load build definition from Dockerfile                                                                                                                                                                                        0.0s
 => => transferring dockerfile: 31B                                                                                                                                                                                                         0.0s
[+] Building 0.4s (4/4)                                                                                                                                                                                                                          
 => [internal] load build definition from Dockerfile                                                                                                                                                                                        0.0s
 => => transferring dockerfile: 31B                                                                                                                                                                                                         0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                       [+] Building 0.6s (4/4)                                                                                                                                                                                                                 
         ansferring context: 2B                                                                                                                                                                                                          => [internal] load build definition from Dockerfile                                                                                                                                                                                    
    0.0slve image config for docker.io/docker/dockerfile:experimental                                                                                                                                                                    => => transferring dockerfile: 31B                                                                                                                                                                                                     
    0.0sED docker-image://docker.io/docker/dockerfile:experimental@sha256:600e5c62eedff338b3f7a0850beb7c05866e0ef27b2d2e8c02aa468e78496ff5                                                                                               => [internal] load .dockerignore                                                                                                                                                                                                       
[+] Building 0.7s (4/4)                                                                                                                                                                                                                 
         ansferring context: 2B                                                                                                                                                                                                          => [internal] load build definition from Dockerfile                                                                                                                                                                                    
    0.0slve image config for docker.io/docker/dockerfile:experimental                                                                                                                                                                    => => transferring dockerfile: 31B                                                                                                                                                                                                     
    0.0sED docker-image://docker.io/docker/dockerfile:experimental@sha256:600e5c62eedff338b3f7a0850beb7c05866e0ef27b2d2e8c02aa468e78496ff5                                                                                               => [internal] load .dockerignore                                                                                                                                                                                                       
[+] Building 0.9s (4/4)                                                                                                                                                                                                                 
         ansferring context: 2B                                                                                                                                                                                                          => [internal] load build definition from Dockerfile                                                                                                                                                                                    
    0.0slve image config for docker.io/docker/dockerfile:experimental                                                                                                                                                                    => => transferring dockerfile: 31B                                                                                                                                                                                                     
    0.0sED docker-image://docker.io/docker/dockerfile:experimental@sha256:600e5c62eedff338b3f7a0850beb7c05866e0ef27b2d2e8c02aa468e78496ff5                                                                                               => [internal] load .dockerignore                                                                                                                                                                                                       
[+] Building 1.0s (4/4)                                                                                                                                                                                                                 
         ansferring context: 2B                                                                                                                                                                                                          => [internal] load build definition from Dockerfile                                                                                                                                                                                    
    0.0slve image config for docker.io/docker/dockerfile:experimental                                                                                                                                                                    => => transferring dockerfile: 31B                                                                                                                                                                                                     
    0.0sED docker-image://docker.io/docker/dockerfile:experimental@sha256:600e5c62eedff338b3f7a0850beb7c05866e0ef27b2d2e8c02aa468e78496ff5                                                                                               => [internal] load .dockerignore                                                                                                                                                                                                       
[+] Building 1.2s (4/4)                                                                                                                                                                                                                 
         ansferring context: 2B                                                                                                                                                                                                          => [internal] load build definition from Dockerfile                                                                                                                                                                                    
    0.0slve image config for docker.io/docker/dockerfile:experimental                                                                                                                                                                    => => transferring dockerfile: 31B                                                                                                                                                                                                     
    0.0sED docker-image://docker.io/docker/dockerfile:experimental@sha256:600e5c62eedff338b3f7a0850beb7c05866e0ef27b2d2e8c02aa468e78496ff5                                                                                               => [internal] load .dockerignore                                                                                                                                                                                                       
[+] Building 1.3s (4/4)                                                                                                                                                                                                                 
         ansferring context: 2B                                                                                                                                                                                                          => [internal] load build definition from Dockerfile                                                                                                                                                                                    
    0.0slve image config for docker.io/docker/dockerfile:experimental                                                                                                                                                                    => => transferring dockerfile: 31B                                                                                                                                                                                                     
    0.0sED docker-image://docker.io/docker/dockerfile:experimental@sha256:600e5c62eedff338b3f7a0850beb7c05866e0ef27b2d2e8c02aa468e78496ff5                                                                                               => [internal] load .dockerignore                                                                                                                                                                                                       
[+] Building 1.4s (4/4) FINISHED                                                                                                                                                                                                        
         ansferring context: 2B                                                                                                                                                                                                          => [internal] load build definition from Dockerfile                                                                                                                                                                                    
    0.0slve image config for docker.io/docker/dockerfile:experimental                                                                                                                                                                    => => transferring dockerfile: 31B                                                                                                                                                                                                     
    0.0sED docker-image://docker.io/docker/dockerfile:experimental@sha256:600e5c62eedff338b3f7a0850beb7c05866e0ef27b2d2e8c02aa468e78496ff5                                                                                               => [internal] load .dockerignore                                                                                                                                                                                                       
    0.0s
 => => transferring context: 2B                                                                                                                                                                                                             0.0s
 => resolve image config for docker.io/docker/dockerfile:experimental                                                                                                                                                                       0.1s
 => CACHED docker-image://docker.io/docker/dockerfile:experimental@sha256:600e5c62eedff338b3f7a0850beb7c05866e0ef27b2d2e8c02aa468e78496ff5                                                                                                  0.0s
failed to solve with frontend dockerfile.v0: failed to solve with frontend gateway.v0: rpc error: code = Unknown desc = failed to create LLB definition: dockerfile parse error line 4: unknown instruction: ECHO

[jin-ahn]

I went back and removed the white space for the heredoc close as shown in my screenshot

As shown in the below screenshot, the Dockerfile does build successfully (ie the heredoc portion works and is not malformed) however as you can see in the 1st screenshot the syntax highlighting and LSP still do not work for anything below the heredoc portion.

I will also note that when I have auto-format on save enabled, the LSP adds the space/indent to the EOF (in my screenshot endscript), which as you pointed out will causes issues when trying to build.

[rcjsuen]

@jin-ahn Please paste your Dockerfile here so I can try to reproduce the syntax highlighting problem.

In the meantime, I am able to reproduce the formatting issue so I'll try to take a look at that. Thank you for bringing this to my attention.

There is a docker.languageserver.formatter.ignoreMultilineInstructions setting which should serve as a workaround that you can toggle as being false but I am not sure how LSP server settings are supposed to be set and configured in your editor.

[jin-ahn]

Thank you for looking into it! Below is my Dockerfile.

# syntax=docker/dockerfile:1.4.1

# ---------------------------------------------------------------------------
# Stage: base_build
# ---------------------------------------------------------------------------

FROM ubuntu:20.04 AS base_build
USER root
RUN  apt-get update \
  &&  apt-get upgrade -y
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y \
  bison \
  build-essential \
  libffi-dev \
  libgdbm-dev \
  libncurses5-dev \
  libreadline-dev \
  libssl-dev \
  libyaml-dev \
  xz-utils \
  zlib1g-dev \
  busybox \
  ca-certificates \
  curl \
  fdupes \
  git \
  libpq-dev \
  python3
RUN apt-get -qy clean \
  && rm -rf /var/lib/apt/lists/*
RUN useradd -d /app app \
  &&    chown app /usr/local/*

# Add script to deduplicate non-empty files in a given directory by hard linking them together.
COPY --chmod=0755 <<-"endscript" /usr/bin/dedup
#!/bin/bash -e
cat > /tmp/dedup-helper <<-"endfunc"
handle_group() {
  src="$1"
  shift
  for dst; do
    cp -fl "$src" "$dst"
  done
  }
endfunc
fdupes -1nqr "$@" | sed -e 's/^/handle_group /' >> /tmp/dedup-helper
busybox ash /tmp/dedup-helper
rm -f /tmp/dedup-helper
endscript

# ---------------------------------------------------------------------------
# Stage: node_build
# ---------------------------------------------------------------------------

FROM base_build AS node_build
RUN --mount=type=bind,source=.nvmrc,target=/tmp/node-version \
  node_version=$(cat /tmp/node-version) \
  &&  echo "uname: $(uname -m)" \
  &&  case "$(uname -m)" in \
  aarch64) node_arch=arm64 ;; \
  x86_64) node_arch=x64 ;; \
  *) echo "unknown arch" \
  exit 1 ;; \
  esac  \
  &&  curl -sfSL https://nodejs.org/dist/v$node_version/node-v$node_version-linux-$node_arch.tar.xz | \
  tar -C /usr/local --strip-components=1 -Jxf - \
  &&  npm install -g yarn

[rcjsuen]

@jin-ahn Thanks for sharing your file. I copied your file and opened it on my online demo site and the syntax highlighting seems okay there... 🤔

[jin-ahn]

@jin-ahn Thanks for sharing your file. I copied your file and opened it on my online demo site and the syntax highlighting seems okay there... 🤔

Hm is there any specific options enabled to the Lsp for your demo site? I run neovim as my editor with no special lsp configs.

[rcjsuen]

@jin-ahn Thanks for sharing your file. I copied your file and opened it on my online demo site and the syntax highlighting seems okay there... 🤔

Hm is there any specific options enabled to the Lsp for your demo site? I run neovim as my editor with no special lsp configs.

No, I don't. There should not be any configuration that would affect how something is highlighted.

[zffocussss]

vscode-docker is not working as expected.when formatting it,the original syntax is broken before

COPY <<-"EOT"
xxx
yyy
EOT
RUN xxx

after

COPY <<-"EOT"
        xxx
        yyy
        EOT
        RUN xxx

[zffocussss]

So I do not think https://github.com/microsoft/vscode-docker/pull/3355 is working now.

[zffocussss]

I am using vscode-docker extension v1.22.1 now

[olrtg]

@jin-ahn if you're using neovim the syntax highlight problem is related to your treesitter parser (nothing to do with this LSP).. maybe they should add the support for that feature (if not supported yet).. or maybe you should update your parser with :TSUpdate dockerfile.

[rcjsuen]

So I do not think microsoft/vscode-docker#3355 is working now.

That change was for RUN instructions. For COPY instructions that was addressed by https://github.com/microsoft/vscode-docker/pull/3955. The more recent versions of the VS Code extension should not have that formatting bug anymore.