moby / buildkit

concurrent, cache-efficient, and Dockerfile-agnostic builder toolkit
https://github.com/moby/moby/issues/34227
Apache License 2.0
8.03k stars 1.12k forks source link

Inconsistent behaviour between command line arguments and config file #4781

Closed fruzitent closed 5 months ago

fruzitent commented 5 months ago

Issue

containerd-cni-binary-dir and containerd-cni-config-path do work, but worker.oci.cniBinaryPath and worker.oci.cniConfigPath do not.

Steps to reproduce

  1. Get nerdctl to generate default configuration it wants [1],

    and replace "cniVersion": "1.0.0" with 0.3.0 because of ? [1] [2].

    ```jsonc // C:\Program Files\containerd\cni\conf\nerdctl-nat.conflist { "cniVersion": "0.3.0", "name": "nat", "nerdctlID": "d919a100ce6b45524d415d52d088d5817587c6dd8c3691b03b8063c44d043523", "nerdctlLabels": { "nerdctl/default-network": "true" }, "plugins": [ { "ipam": { "routes": [ { "gateway": "10.4.0.1" } ], "subnet": "10.4.0.0/24", "type": "" }, "master": "Ethernet", "type": "nat" } ] } ```
  2. Point buildkitd to use this via command line arguments.

    $ buildkitd `
        --containerd-cni-binary-dir "${Env:ProgramFiles}\containerd\cni\bin\" `
        --containerd-cni-config-path "${Env:ProgramFiles}\containerd\cni\conf\nerdctl-nat.conflist" `
        ;
    {"level":"debug","msg":"creating new network namespace cwc38k39ai9prgrbkc6xj53td","time":"2024-03-19T05:53:12+02:00"}

    It works!

  3. Put the same settings in config file [1].

    # C:\ProgramData\buildkitd\buildkitd.toml
    [worker.oci]
    cniBinaryPath = "C:\\Program Files\\containerd\\cni\\bin"
    cniConfigPath = "C:\\Program Files\\containerd\\cni\\conf\\nerdctl-nat.conflist"

    It was able to use debug and log.format options successfully, but suddenly cni stops working.

    Neither of the cmd or pwsh variables, full path with forward or backward slashes - none of this matters, these fields are simply ignored for some reason.

  4. Copy nat.exe to C:\ProgramData\buildkitd\bin [1], as well as regular cni conf [2] [3], instead of conflist (it used to work when run from the command line).

    ```jsonc // C:\ProgramData\buildkitd\cni.json { "cniVersion": "0.3.0", "name": "nat", "type": "nat", "master": "Ethernet", "ipam": { "subnet": "10.4.0.0/24", "routes": [ { "gateway": "10.4.0.1" } ] }, "capabilities": { "portMappings": true, "dns": true } } ```
    $ buildkitd
    {"level":"debug","msg":"creating new network namespace 9yptgzxgas0nvj7ct3dev27lh","time":"2024-03-19T06:09:57+02:00"}

    It works!

Version

$ buildkitd --version
buildkitd github.com/moby/buildkit v0.13.1 2ae42e0c0c793d7d66b7a23424af6fd6c2f9c8f3
$ containerd --version
containerd github.com/containerd/containerd v1.7.14 dcf2847247e18caba8dce86522029642f60fe96b
$ nerdctl --version
nerdctl version 1.7.5
tonistiigi commented 5 months ago

equivalent to containerd-cni-binary-dir would be

[worker.containerd]
cniBinaryPath =