Simple image building with BuildImageFromDockerfileAsync does not build an image

[Stadzior]

Output of dotnet --info:

.NET SDK:
 Version:   7.0.100
 Commit:    e12b7af219

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.19044
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\7.0.100\

Host:
  Version:      7.0.0
  Architecture: x64
  Commit:       d099f075e4

.NET SDKs installed:
  2.1.700 [C:\Program Files\dotnet\sdk]
  2.1.701 [C:\Program Files\dotnet\sdk]
  2.2.300 [C:\Program Files\dotnet\sdk]
  2.2.301 [C:\Program Files\dotnet\sdk]
  6.0.100-preview.1.21103.13 [C:\Program Files\dotnet\sdk]
  7.0.100 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.All 2.1.11 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.12 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.11 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.12 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.0-preview.1.21103.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.11 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 7.0.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.11 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.12 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.0-preview.1.21102.12 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.0-preview.5.21301.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.11 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 7.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 6.0.0-preview.1.21103.5 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 6.0.11 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 7.0.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Other architectures found:
  arm64 [C:\Program Files\dotnet]
    registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\arm64\InstallLocation]
  x86   [C:\Program Files (x86)\dotnet]
    registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x86\InstallLocation]

Environment variables:
  Not set

global.json file:
  Not found

Learn more:
  https://aka.ms/dotnet/info

Download .NET:
  https://aka.ms/dotnet/download

What version of Docker.DotNet?:

3.125.12

Steps to reproduce the issue: Just run below code as a ConsoleApp (ofc. you need docker deamon running):

using Docker.DotNet;
using Docker.DotNet.Models;
using System.Text;
using ICSharpCode.SharpZipLib.Tar;

var dockerEngineUrl = new Uri("npipe://./pipe/docker_engine");
using var client = new DockerClientConfiguration(dockerEngineUrl).CreateClient();

const string imageName = "myimage";
const string dockerfilePath = "./../../../source";
await BuildImage(client, imageName, dockerfilePath);

async Task BuildImage(IDockerClient client, string imageName, string dockerFilePath, bool rebuild = false)
{
    if (!rebuild)
        rebuild = !await ImageAlreadyExists(client, imageName); //If should not rebuild existing image, check if it exists. If yes then do not build.

    if (rebuild)
    {
        Console.WriteLine($"Image named '{imageName}' not found. Building image from {dockerFilePath}. {DateTime.Now.ToLongTimeString()}");

        var imageBuildParameters = new ImageBuildParameters
        {
            Dockerfile = dockerFilePath
        };

        var progress = new Progress<JSONMessage>();
        progress.ProgressChanged += (sender, e) =>
        {
            Console.WriteLine($"From: {e.From}");
            Console.WriteLine($"Status: {e.Status}");
            Console.WriteLine($"Stream: {e.Stream}");
            Console.WriteLine($"ID: {e.ID}");
            Console.WriteLine($"Progress: {e.ProgressMessage}");
            Console.WriteLine($"Error: {e.ErrorMessage}");
        };

        var stream = CreateTarFileForDockerfileDirectory(dockerFilePath);

        await client.Images.BuildImageFromDockerfileAsync(imageBuildParameters, stream, null, null, progress);

        Console.WriteLine($"Image named '{imageName}' built. {DateTime.Now.ToLongTimeString()}");
    }
}

async Task<bool> ImageAlreadyExists(IDockerClient client, string imageName)
{
    var listImagesParameters = new ImagesListParameters { All = true };
    var images = await client.Images.ListImagesAsync(listImagesParameters);
    return images.Any(image => image.RepoTags.Contains(imageName));
}

Stream CreateTarFileForDockerfileDirectory(string directory)
{
    var stream = new MemoryStream();
    var filePaths = Directory.GetFiles(directory, "*.*", SearchOption.AllDirectories);

    using var archive = new TarOutputStream(stream, Encoding.UTF8);

    //Prevent the TarOutputStream from closing the underlying memory stream when done
    archive.IsStreamOwner = false;

    //Add files to tar archive
    foreach (var file in filePaths)
    {
        var tarName = Path.GetFileNameWithoutExtension(file);

        var entry = TarEntry.CreateTarEntry(tarName);
        using var fileStream = File.OpenRead(file);
        entry.Size = fileStream.Length;
        archive.PutNextEntry(entry);

        var localBuffer = new byte[32 * 1024];

        while (true)
        {
            var numberOfBytesSavedToBuffer = fileStream.Read(localBuffer, 0, localBuffer.Length);
            if (numberOfBytesSavedToBuffer <= 0)
                break;

            archive.Write(localBuffer, 0, numberOfBytesSavedToBuffer);
        }
        archive.CloseEntry();
    }

    archive.Close();

    stream.Position = 0;
    return stream;
}

Source directory content: Dockerfile:

FROM alpine
ADD test.txt .
ENTRYPOINT ["/bin/cat", "test.txt"]

test.txt:

Hello from test.txt

What actually happened?: BuildImageFromDockerfileAsync went through without an error and there is no "myimage" visible under docker image list.

What did you expect to happen?: "myimage" visible under docker image list

Additional information:

[HofmeisterAn]

At least your dockerfilePath is wrong. ImageBuildParameters.Dockerfile is relative to the Dockerfile inside the tarball. It is also the file path to the Dockerfile, not just the parent directory. BuildImageFromDockerfileAsync runs fine in our tests.

[Stadzior]

@HofmeisterAn right. Removing Dockerfile from ImageBuildParameters leaded me to another error: ADD failed: file not found in build context or excluded by .dockerignore: stat test.txt: file does not exist. Can you please give me an example on how you build your tarball? I bet my method is somehow invalid and that's why test.txt is not found.

[HofmeisterAn]

Accordingly to your other questions in the past days you might wanna take a look at Testcontainers. It looks like it covers everything you are trying to implement.

Can you please give me an example on how you build your tarball?

Here you go.

[mikeblas]

ImageBuildParameters.Dockerfile is relative to the Dockerfile inside the tarball. It is also the file path to the Dockerfile

Can you explain what that means? How can a path to a file in a file system be relative to a file in a tarball?

BuildImageFromDockerfileAsync runs fine in our tests.

Where can I find these tests? I don't see any calls to any BuildImageFromDockerfileAsync() override in the test project.

[HofmeisterAn]

Can you explain what that means? How can a path to a file in a file system be relative to a file in a tarball?

To build a Docker image, the Docker daemon receives a tarball that contains all necessary files. The Dockerfile arg represents the path inside the tarball.

Where can I find these tests? I don't see any calls to any BuildImageFromDockerfileAsync() override in the test project.

It is part of Testcontainers for .NET.

[mikeblas]

Thanks for the clarification! Also, sorry -- I thought you were referring to the Docker.DotNet tests.