search

containers / podman

Podman: A tool for managing OCI containers and pods.
https://podman.io
Apache License 2.0
23.79k stars 2.42k forks source link

Multiple build context fails on Windows #18840

Closed leadoflife closed 2 months ago

leadoflife commented 1 year ago

Issue Description

Using the argument --build-context fails on Windows with Podman + WSL.

> podman build --build-context a=1st .
STEP 1/5: FROM docker.io/sonatype/nexus3:latest
STEP 2/5: COPY --from=a . /usr/src/a/
Error: building at STEP "COPY --from=a . /usr/src/a/": checking on sources under "/home/user/C:\\Users\\my_username\\src\\tmp\\podman\\multi-ctx\\1st": error in copier subprocess: changing to intended-new-root directory "/home/user/C:\\Users\\my_username\\src\\tmp\\podman\\multi-ctx\\1st": chdir /home/user/C:\Users\my_username\src\tmp\podman\multi-ctx\1st: no such file or directory

With debug logging...

> podman build --debug --build-context a=1st .
time="2023-06-09T12:00:21-04:00" level=info msg="podman filtering at log level debug"
time="2023-06-09T12:00:21-04:00" level=debug msg="Called build.PersistentPreRunE(podman build --log-level debug --build-context a=1st .)"
time="2023-06-09T12:00:21-04:00" level=debug msg="SSH Ident Key \"C:\\\\Users\\\\my_username\\\\.ssh\\\\podman-machine-default\" SHA256:Q1wkbCLO3knjjfrzlF9jKLP/2J5ffPLSxPhNQNyYUnQ ssh-ed25519"
time="2023-06-09T12:00:21-04:00" level=debug msg="DoRequest Method: GET URI: http://d/v4.5.0/libpod/_ping"
time="2023-06-09T12:00:21-04:00" level=debug msg="Loading registries configuration \"/etc/containers/registries.conf\""
time="2023-06-09T12:00:21-04:00" level=debug msg="Found credentials for ghcr.io in credential helper containers-auth.json in file C:\\Users\\my_username\\.config\\containers\\auth.json"
time="2023-06-09T12:00:21-04:00" level=debug msg="DoRequest Method: POST URI: http://d/v4.5.0/libpod/build"
STEP 1/2: FROM docker.io/library/hello-world:latest
STEP 2/2: COPY --from=a . /usr/src/a/
Error: building at STEP "COPY --from=a . /usr/src/a/": checking on sources under "/home/user/C:\\Users\\my_username\\src\\tmp\\podman\\multi-ctx\\1st": error in copier subprocess: changing to intended-new-root directory "/home/user/C:\\Users\\my_username\\src\\tmp\\podman\\multi-ctx\\1st": chdir /home/user/C:\Users\my_username\src\tmp\podman\multi-ctx\1st: no such file or directory

time="2023-06-09T12:00:21-04:00" level=debug msg="Shutting down engines"

Steps to reproduce the issue

Steps to reproduce the issue

  1. On Windows (11 Pro)
  2. With Podman installed using WSL (v2)
  3. Attempt to build using the --build-context argument

Both cmd and "Git BASH" produce errors. Using both helps rule out "pure" Linux path vs. Windows path issues.

Describe the results you received

Given a file and directory structure of:

Single, alternative build context

Where, "./Containerfile" contains steps for a single, alternative context build:

FROM docker.io/library/hello-world:latest
COPY --from=a . /usr/src/a/

Using cmd...

> podman build --build-context a=1st .
STEP 1/2: FROM docker.io/library/hello-world:latest
STEP 2/2: COPY --from=a . /usr/src/a/
Error: building at STEP "COPY --from=a . /usr/src/a/": checking on sources under "/home/user/C:\\Users\\my_username\\src\\tmp\\podman\\multi-ctx\\1st": error in copier subprocess: changing to intended-new-root directory "/home/user/C:\\Users\\my_username\\src\\tmp\\podman\\multi-ctx\\1st": chdir /home/user/C:\Users\my_username\src\tmp\podman\multi-ctx\1st: no such file or directory

Using "Git BASH"...

$ podman build --build-context a=./1st/ .
STEP 1/2: FROM docker.io/library/hello-world:latest
STEP 2/2: COPY --from=a . /usr/src/a/
Error: building at STEP "COPY --from=a . /usr/src/a/": checking on sources under "/home/user/C:\\Users\\iseri\\src\\tmp\\podman\\multi-ctx\\1st": error in copier subprocess: changing to intended-new-root directory "/home/user/C:\\Users\\iseri\\src\\tmp\\podman\\multi-ctx\\1st": chdir /home/user/C:\Users\iseri\src\tmp\podman\multi-ctx\1st: no such file or directory

Multiple context build

Where, "./Containerfile" contains steps for a multiple context build:

FROM docker.io/library/hello-world:latest

COPY --from=a . /usr/src/a/
COPY --from=b . /usr/src/b/
COPY --from=c . /usr/src/c/
COPY --from=sd . /usr/src/sd/

Using cmd...

> podman build --build-context a=1st b=2nd c=3rd sd=..\subdir .

Using "Git BASH"...

$ podman build --build-context a=./1st/ b=./2nd/ c=./3rd/ sd=../subdir/ .

... both cmd and "Git BASH" fail on the first entry, but this is included as an important use case that proves 1) multiple build contexts as well as a path up and back down through a different directory structure.

Describe the results you expected

Files get copied successfully from each aliased build context.

podman info output

host:
  arch: amd64
  buildahVersion: 1.29.0
  cgroupControllers: []
  cgroupManager: cgroupfs
  cgroupVersion: v1
  conmon:
    package: conmon-2.1.7-2.fc37.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.7, commit: '
  cpuUtilization:
    idlePercent: 99.86
    systemPercent: 0.07
    userPercent: 0.07
  cpus: 16
  databaseBackend: ""
  distribution:
    distribution: fedora
    variant: container
    version: "37"
  eventLogger: journald
  hostname: LAPTOP-E4NFDVCK
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 524288
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 524288
      size: 65536
  kernel: 5.15.90.1-microsoft-standard-WSL2
  linkmode: dynamic
  logDriver: journald
  memFree: 2190303232
  memTotal: 16668450816
  networkBackend: netavark
  ociRuntime:
    name: crun
    package: crun-1.8.3-2.fc37.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.8.3
      commit: 59f2beb7efb0d35611d5818fd0311883676f6f7e
      rundir: /run/user/1000/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL
  os: linux
  remoteSocket:
    exists: true
    path: /run/user/1000/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: true
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.2.0-8.fc37.x86_64
    version: |-
      slirp4netns version 1.2.0
      commit: 656041d45cfca7a4176f6b7eed9e4fe6c11e8383
      libslirp: 4.7.0
      SLIRP_CONFIG_VERSION_MAX: 4
      libseccomp: 2.5.3
  swapFree: 4294967296
  swapTotal: 4294967296
  uptime: 136h 56m 13.00s (Approximately 5.67 days)
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  volume:
  - local
registries:
  search:
  - docker.io
store:
  configFile: /home/user/.config/containers/storage.conf
  containerStore:
    number: 1
    paused: 0
    running: 1
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /home/user/.local/share/containers/storage
  graphRootAllocated: 1081101176832
  graphRootUsed: 34710867968
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 121
  runRoot: /run/user/1000/containers
  transientStore: false
  volumePath: /home/user/.local/share/containers/storage/volumes
version:
  APIVersion: 4.4.4
  Built: 1680521485
  BuiltTime: Mon Apr  3 07:31:25 2023
  GitCommit: ""
  GoVersion: go1.19.7
  Os: linux
  OsArch: linux/amd64
  Version: 4.4.4

Podman in a container

No

Privileged Or Rootless

Rootless

Upstream Latest Release

Yes

Additional environment details

Windows System

Edition Windows 11 Pro
Version 22H2
Installed on    ‎9/‎27/‎2022
OS build    22621.1702
Serial number   MP1WGZ82
Experience  Windows Feature Experience Pack 1000.22641.1000.0

System specifications

Processor   Intel(R) Core(TM) i9-10980HK CPU @ 2.40GHz   3.10 GHz
Installed RAM   32.0 GB (31.8 GB usable)
System type 64-bit operating system, x64-based processor

Additional information

No response

leadoflife commented 1 year ago

Different, but related issue #10100 with a similar file system issue and notes on buildah as well as reproducing with rootless and root. Though no obvious hints (to me) on how to fix are apparent. The notes on RHEL and Alpine make me wonder if the Fedora virtual that Podman is running on in WSL might have a similar kernal and/or buildah issue. But, the error seems to be in mapping the Windows file system path to the underlying Linux paths... so, it's likely in the Podman path mapping code.

leadoflife commented 1 year ago

I'm not a RedHat subscriber, but the paywall solution behind this article may prove useful if someone is a subscriber: https://access.redhat.com/solutions/6958392

leadoflife commented 1 year ago

Workaround

While the above is still a valid bug, a work-around to enable multi-context builds is to copy the files / directories into a tagged image and then use the aliased container build context to reference those files.

Example:

"./shared/Containerfile"

FROM docker.io/library/debian:11.7-slim
COPY ./1st/ /usr/src/a/
COPY ./2nd/ /usr/src/b/

VOLUME [ "/usr/src/a/", "/usr/src/b/" ]

"./multi-context-build/Containerfile"

FROM docker.io/library/debian:11.7-slim
COPY --from=libraries /usr/src/a/ /usr/src/a/
COPY --from=libraries /usr/src/b/ /usr/src/b/
  1. Build the shared file system into an image with the command cd ./shared/ && podman build --tag my-shared .
  2. Use the shared file system image in another multi-context build with the command cd ./multi-context-build/ && podman build --build-context libraries=container-image://localhost/my-shared .

Astute readers may note there are simpler methods to structure the volume and to COPY, but this contrived example was intended to suggest that each shared directory could be named anything and could use any potential path on the file system.

Luap99 commented 1 year ago

@flouthoc PTAL

github-actions[bot] commented 1 year ago

A friendly reminder that this issue had no activity for 30 days.

gavenkoa commented 11 months ago

Facing with same error:

Error: building at STEP "COPY --from=ext . .": checking on sources under "/C:\\Users\\user\\work\\proj\\ext": error in copier subprocess: changing to intended-new-root directory "/C:\\Users\\user\\work\\proj\\ext": chdir /C:\Users\user\work\proj\ext: no such file or directory

when I provided --build-context=ext=../ext - to escape from the build context when referring to external files. The command is:

COPY --from=ext . .

Windows 10. podman.exe version 4.7.1.

gavenkoa commented 11 months ago

Seems handling of Windows paths is broken for podman cp too: https://stackoverflow.com/questions/77085191/how-can-i-copy-paste-files-into-a-wsl-podman-container

l0rd commented 2 months ago

This issue has been fixed when the machine provider is WSL (the default) but it's still failing when the machine provider is Hyper-V. I am closing it as there is an Hyper-V dedicated issue to track that problem.