search

moby / buildkit

concurrent, cache-efficient, and Dockerfile-agnostic builder toolkit
https://github.com/moby/moby/issues/34227
Apache License 2.0
8.03k stars 1.12k forks source link

Support `RUN --mount=type=ssh` for Windows containers #4837

Open AnastaZIuk opened 5 months ago

AnastaZIuk commented 5 months ago

Hi, I've been tracking Windows containers support issue for some time and after reading this comment I wanted to check it out.

Unfortunately I have encountered issues when trying to use RUN --mount=type=ssh instruction, I'm aware Windows containers + Buildkit is experimental and in development phase so I would like to report a possible bug.

My environment is

amd64 arch Windows 11 Pro OS 10.0.22631 Build 22631
buildctl github.com/moby/buildkit v0.13.1 2ae42e0c0c793d7d66b7a23424af6fd6c2f9c8f3
containerd github.com/containerd/containerd v1.7.13 7c3aca7a610df76212171d200ca3811ff6096eb8

Reproduction

build an image with ssh host proxy

PS D:\test> buildctl build --frontend=dockerfile.v0 --local context=. --local dockerfile=. --output type=image,name=test,push=false --ssh default [+] Building 0.6s (5/5) FINISHED => [internal] load build definition from Dockerfile 0.0s => => transferring dockerfile: 174B 0.0s => [internal] load metadata for mcr.microsoft.com/windows/nanoserver:ltsc2022 0.3s => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => CACHED [stage-0 1/2] FROM mcr.microsoft.com/windows/nanoserver:ltsc2022@sha256:6223f3a17f2652064c56ca55aaef866177d58d6a1c526636ac2120685df5466d 0.0s => => resolve mcr.microsoft.com/windows/nanoserver:ltsc2022@sha256:6223f3a17f2652064c56ca55aaef866177d58d6a1c526636ac2120685df5466d 0.0s => ERROR [stage-0 2/2] RUN --mount=type=ssh ssh -T git@github.com 0.0s

[stage-0 2/2] RUN --mount=type=ssh ssh -T git@github.com:

Dockerfile:3

1 | FROM mcr.microsoft.com/windows/nanoserver:ltsc2022 2 | USER ContainerAdministrator 3 | >>> RUN --mount=type=ssh ssh -T git@github.com 4 | CMD ["cmd"] 5 |

error: failed to solve: failed to compute cache key: failed to calculate checksum of ref okff89cu84wigb87xavgu4b8e::uohocwd1ds4k87bophhalgrag: failed to walk C:\Users\arekl\AppData\Local\Temp\buildkit-mount3862397659\WcSandboxState: open C:\Users\arekl\AppData\Local\Temp\buildkit-mount3862397659\WcSandboxState: Access is denied. PS D:\test>

I have also tried socket syntax <id>=<socket> with SSH_AUTH_SOCK env + all above with git bash shell but with no luck, I get the same errors.

profnandaa commented 4 months ago

@AnastaZIuk -- thanks for checking it out! ~are you running buildctl on an elevated PowerShell too?~ nvm, I can repro it; investigating.

AnastaZIuk commented 4 months ago

@AnastaZIuk -- thanks for checking it out! ~are you running buildctl on an elevated PowerShell too?~ nvm, I can repro it; investigating.

yes I'm running with elevated PS, cmd behaves the same way when launched in elevated mode

profnandaa commented 4 months ago

Just posting here my prelimimary findings so far:

> github.com/moby/buildkit/cache/contenthash.(*cacheContext).scanPath.func2() C:/dev/container-core/buildkit/cache/contenthash/checksum.go:1031 (hits goroutine(595):1 total:1) (PC: 0x27f478a)
  1026:                 return err
  1027:         }
  1028:
  1029:         err = filepath.Walk(parentPath, func(itemPath string, fi os.FileInfo, err error) error {
  1030:                 if err != nil {
=>1031:                         return errors.Wrapf(err, "failed to walk %s", itemPath)
  1032:                 }
  1033:                 rel, err := filepath.Rel(mp, itemPath)
  1034:                 if err != nil {
  1035:                         return err
  1036:                 }
(dlv) p err
error(*io/fs.PathError) *{
        Op: "open",
        Path: "C:\\Users\\nandaa\\AppData\\Local\\Temp\\buildkit-mount1136466203\\Syst...+21 more",
        Err: error(syscall.Errno) ERROR_ACCESS_DENIED (5),}
(dlv) config max-string-len 1024
(dlv) p err.Path
"C:\\Users\\nandaa\\AppData\\Local\\Temp\\buildkit-mount1136466203\\System Volume Information"
(dlv)

I'm suspicious about the "System Volume Information" suffix in the Path. ./cc. @gabriel-samfira

profnandaa commented 4 months ago

I see, it's because C:\System Volume Information directory is a protected directory and can't be walked without specific ACLs. Checking how this has been treated before elsewhere or should be skipped.

gabriel-samfira commented 4 months ago

@profnandaa yes, this has been handled elsewhere as well, for most of the same reasons:

https://github.com/containerd/continuity/pull/113 https://github.com/containerd/continuity/pull/212 https://github.com/containerd/continuity/pull/219

I have not looked at the ssh mount code in buildkitd, but if the containerd snapshotter is used, it should not generate this error. Will try to allocate some time to look at this this week.