Docket.DotNet Build Image has undocumented limitations

[tyb-dev]

Output of dotnet --info:

.NET SDK:
 Version:   7.0.101
 Commit:    bb24aafa11

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.19044
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\7.0.101\

Host:
  Version:      7.0.1
  Architecture: x64
  Commit:       97203d38ba

.NET SDKs installed:
  6.0.100 [C:\Program Files\dotnet\sdk]
  7.0.101 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.17 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.12 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 7.0.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.17 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.12 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 7.0.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 5.0.17 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 6.0.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 6.0.12 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 7.0.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Other architectures found:
  x86   [C:\Program Files (x86)\dotnet]
    registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x86\InstallLocation]

Environment variables:
  Not set

global.json file:
  Not found

Learn more:
  https://aka.ms/dotnet/info

Download .NET:
  https://aka.ms/dotnet/download

What version of Docker.DotNet?:

3.125.14

Steps to reproduce the issue: Dockerfile

FROM debian
WORKDIR /data
COPY --chmod=+x . /data

Code (ConsoleApp calling BuildImageFromDockerfileAsync)

using Docker.DotNet;
using Docker.DotNet.Models;
using System.Text;
using ICSharpCode.SharpZipLib.Tar;

using var client = new DockerClientConfiguration(new Uri("npipe://./pipe/docker_engine"))
    .CreateClient();

await BuildDockerImage(client, ".");

static async Task BuildDockerImage(DockerClient client, string dockerFileDirectory)
{
    // Create an ImageBuildParameters object
    var buildParameters = new ImageBuildParameters
    {
        Dockerfile = "Dockerfile",
        Remove = true, // Remove intermediate containers after a successful build
        Tags = new[] { "yourimagename:tag" }, // Set your image name and tag
        ForceRemove = true, // Always remove intermediate containers, even upon failure
        NoCache = true // Do not use the cache when building the image,
    };

    // Read the dockerfile content
    var dockerFileContent = File.ReadAllText(Path.Combine(dockerFileDirectory, buildParameters.Dockerfile));

    // Create a tarball with the Dockerfile
    var tarball = new FileStream("new.tar", FileMode.Create);
    await using (var tarArchive = new TarOutputStream(tarball))
    {
        var tarEntry = TarEntry.CreateTarEntry("Dockerfile");
        var fileBytes = Encoding.UTF8.GetBytes(dockerFileContent);
        tarEntry.Size = fileBytes.Length;
        tarArchive.PutNextEntry(tarEntry);
        tarArchive.Write(fileBytes, 0, fileBytes.Length);
        tarArchive.CloseEntry();
    }

    tarball = new FileStream("new.tar", FileMode.Open);

    // Build the image
    await using (var responseStream = await client.Images.BuildImageFromDockerfileAsync(tarball, buildParameters))
    {
        // Read the response from the Docker API
        using var reader = new StreamReader(responseStream);
        while (!reader.EndOfStream)
        {
            var line = await reader.ReadLineAsync();
            Console.WriteLine(line);
        }
    }
}

What actually happened?:

{"errorDetail":{"message":"the --chmod option requires BuildKit. Refer to https://docs.docker.com/go/buildkit/ to learn how to build images with BuildKit enabled"},"error":"the --chmod option requires BuildKit. Refer to https://docs.docker.com/go/buildkit/ to learn how to build images with BuildKit enabled"}

What did you expect to happen?: Image is created equivalent to doing docker build on the command line.

Additional information: https://github.com/moby/moby/issues/41711#issuecomment-733281140

[cdhanna]

I am wondering if this is the same problem causing me difficulties related to multi-architecture builds. If I try to do this in my Dockerfile, I'd expect to get the $BUILDPLATFORM from docker, similar to how it appears via the docker CLI.

FROM --platform=$BUILDPLATFORM mcr.microsoft.com/dotnet/sdk:8.0-alpine AS build-env

According to the docs, that only appears with buildkit. https://docs.docker.com/reference/dockerfile/#automatic-platform-args-in-the-global-scope

Any ETA on a fix for this? I would love to be able to use buildKit.

[RafalOsieka]

I have similar issue when using RUN --mount... in the Dockerfile - https://docs.docker.com/build/building/secrets/#target

the --mount option requires BuildKit. Refer to https://docs.docker.com/go/buildkit/ to learn how to build images with BuildKit enabled

[cdhanna]

back in May I took a shot at forking this library and using v2 of the docker API; so as to unlock buildkit support. However, when you do that, and you request buildkit in the v2 API, it gives you back logs as the encoded protobuf structures directly from the GO implementation. This library doesn't have the protobuf decoders installed, so its essentially impossible to decode the log messages to gather any sort of progress information. To be honest, it was a pain, and I gave up and started invoking docker via the CLI instead.

[alexaka1]

I landed here after not understanding why TestContainers was not working for me. It is crucial to add Buildx support. Docker went all in on it, along with containerd. .Net is also multiplatform, the ability to test multiplatform images is very relevant now.