podman build --secret help doesn't match docs and does not work for type=env

apollo13 commented 3 months ago

Issue Description

I am trying to pass some secret into a container build without leaving any traces in the container itself. Playing around I tried:

podman build --secret secret,type=env,target=ASD -f Containerfile.test .

which complains:

Error: creating build executor: incorrect secret flag format: should be --secret id=foo,src=bar[,env=ENV,type=file|env]

So far so good, prefixing with id= successfully builds but doesn't expose the env to the container. The docs seem to suggest that this won't work anyways https://docs.podman.io/en/latest/markdown/podman-build.1.html#secret-id-id-src-path and apparently only supported for podman run/create: https://docs.podman.io/en/latest/markdown/podman-run.1.html#secret-secret-opt-opt

So I am assuming that podman build uses the same "parsing logic" while not really supporting all of it.

Ideally I'd like to be able to set a transient env variable during a container build.

Steps to reproduce the issue

$ cat Containerfile.test
FROM python:3.12-slim
RUN echo $ASD
$ podman build --no-cache --secret id=secret,type=env,target=ASD -f Containerfile.test . 
STEP 1/2: FROM python:3.12-slim
STEP 2/2: RUN echo $ASD

COMMIT
--> 2dd962b49c90
2dd962b49c90baa044a1dcb1887baf9c546622dcfa5f4e73c1270df8236630fb

Describe the results you received

$ASD is empty inside the build context

Describe the results you expected

I'd love to see the content of the secret

podman info output

host:
  arch: amd64
  buildahVersion: 1.36.0
  cgroupControllers:
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.10-1.fc40.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.10, commit: '
  cpuUtilization:
    idlePercent: 97.82
    systemPercent: 0.67
    userPercent: 1.51
  cpus: 16
  databaseBackend: sqlite
  distribution:
    distribution: fedora
    variant: workstation
    version: "40"
  eventLogger: journald
  freeLocks: 2045
  hostname: apollo13
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 524288
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 524288
      size: 65536
  kernel: 6.9.9-200.fc40.x86_64
  linkmode: dynamic
  logDriver: journald
  memFree: 4261511168
  memTotal: 32399306752
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    dns:
      package: aardvark-dns-1.11.0-1.fc40.x86_64
      path: /usr/libexec/podman/aardvark-dns
      version: aardvark-dns 1.11.0
    package: netavark-1.11.0-1.fc40.x86_64
    path: /usr/libexec/podman/netavark
    version: netavark 1.11.0
  ociRuntime:
    name: crun
    package: crun-1.15-1.fc40.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.15
      commit: e6eacaf4034e84185fd8780ac9262bbf57082278
      rundir: /run/user/1000/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL
  os: linux
  pasta:
    executable: /usr/bin/pasta
    package: passt-0^20240624.g1ee2eca-1.fc40.x86_64
    version: |
      pasta 0^20240624.g1ee2eca-1.fc40.x86_64
      Copyright Red Hat
      GNU General Public License, version 2 or later
        <https://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
      This is free software: you are free to change and redistribute it.
      There is NO WARRANTY, to the extent permitted by law.
  remoteSocket:
    exists: false
    path: /run/user/1000/podman/podman.sock
  rootlessNetworkCmd: pasta
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: false
  slirp4netns:
    executable: ""
    package: ""
    version: ""
  swapFree: 8589930496
  swapTotal: 8589930496
  uptime: 7h 55m 7.00s (Approximately 0.29 days)
  variant: ""
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - docker.io
store:
  configFile: /home/florian/.config/containers/storage.conf
  containerStore:
    number: 1
    paused: 0
    running: 1
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /home/florian/.local/share/containers/storage
  graphRootAllocated: 536854134784
  graphRootUsed: 288820391936
  graphStatus:
    Backing Filesystem: btrfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Supports shifting: "false"
    Supports volatile: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 417
  runRoot: /run/user/1000/containers
  transientStore: false
  volumePath: /home/florian/.local/share/containers/storage/volumes
version:
  APIVersion: 5.1.1
  Built: 1717459200
  BuiltTime: Tue Jun  4 02:00:00 2024
  GitCommit: ""
  GoVersion: go1.22.3
  Os: linux
  OsArch: linux/amd64
  Version: 5.1.1



### Podman in a container

No

### Privileged Or Rootless

None

### Upstream Latest Release

No

### Additional environment details

_No response_

### Additional information

_No response_

rhatdan commented 3 months ago

So you would expect that each RUN line would have access to the secret environment variable.

rhatdan commented 3 months ago

@flouthoc @nalind WDYT?

apollo13 commented 3 months ago

@rhatdan Yes, that would be nice. To elaborate: I am trying to set environment variables for common package managers (like python's pip and npm) so that the container builds use our internal mirrors without having to manually add support for that inside the Containerfile (currently I have a build-arg ala: ARG PIP_PYPI_URL=https://pypi.org/simple but that leaks into the container history if I override it from the outside -- unless I use it in an intermediate container in multi stage builds.).

I fully understand though if that stays only supported for podman run/create. Nevertheless it would be really nice to have and might somewhat improve security as well.

rhatdan commented 3 months ago

Yes I think this is reasonable.

rhatdan commented 3 months ago

@ashley-cui WDYT?

flouthoc commented 3 months ago

@apollo13 I think --mount=type=secret only exposes secret for a specific RUN and not creates secret for entire container, it will expose it on the dst path but will be transient in nature and you can manually export it to your preferred env variable.

Please feel free to close the issue if you think above should suffice your use-case.

apollo13 commented 3 months ago

@flouthoc Yes, that works, but requires cooperation from inside the docker file. With my proposed solution I could "enforce" a company wide pypi mirror (etc) without manually touching all the dockerfiles. Also the fallback is rather ugly if you mount something in RUN it probably has to exist in some form.

rhatdan commented 3 months ago

I think setting the environment variable for the length of the RUN command would be a good idea.

github-actions[bot] commented 2 months ago

A friendly reminder that this issue had no activity for 30 days.

ralphie79 commented 1 week ago

Ideally I'd like to be able to set a transient env variable during a container build.

+1 I've run into a similar use case with podman build in the context of a CI engine which is already making secrets available as environment variables. To get them into podman, they first have to be written to files to be subsequently imported as secrets.

containers / podman