[Bug]:

[davidkeaveny]

Testcontainers version

3.5.0

Using the latest Testcontainers version?

Yes

Host OS

Linux

Host arch

x86

.NET version

6.0.22

Docker version

Client:
 Cloud integration: v1.0.35-desktop+001
 Version:           24.0.5
 API version:       1.43
 Go version:        go1.20.6
 Git commit:        ced0996
 Built:             Fri Jul 21 20:36:24 2023
 OS/Arch:           windows/amd64
 Context:           default

Server: Docker Desktop 4.22.1 (118664)
 Engine:
  Version:          24.0.5
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.6
  Git commit:       a61e2b4
  Built:            Fri Jul 21 20:35:45 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.21
  GitCommit:        3dce8eb055cbb6872793272b4f20ed16117344f8
 runc:
  Version:          1.1.7
  GitCommit:        v1.1.7-0-g860f061
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Docker info

Client:
 Version:    24.0.5
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.11.2-desktop.1
    Path:     C:\Program Files\Docker\cli-plugins\docker-buildx.exe
  compose: Docker Compose (Docker Inc.)
    Version:  v2.20.2-desktop.1
    Path:     C:\Program Files\Docker\cli-plugins\docker-compose.exe
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-dev.exe
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.20
    Path:     C:\Program Files\Docker\cli-plugins\docker-extension.exe
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v0.1.0-beta.6
    Path:     C:\Program Files\Docker\cli-plugins\docker-init.exe
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-sbom.exe
  scan: Docker Scan (Docker Inc.)
    Version:  v0.26.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-scan.exe
  scout: Command line tool for Docker Scout (Docker Inc.)
    Version:  0.20.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-scout.exe

Server:
 Containers: 3
  Running: 3
  Paused: 0
  Stopped: 0
 Images: 20
 Server Version: 24.0.5
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 3dce8eb055cbb6872793272b4f20ed16117344f8
 runc version: v1.1.7-0-g860f061
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
 Kernel Version: 5.10.102.1-microsoft-standard-WSL2
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 15.55GiB
 Name: docker-desktop
 ID: 8b458123-8e9c-48d4-a740-8b2cb83c2be3
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No blkio throttle.read_bps_device support
WARNING: No blkio throttle.write_bps_device support
WARNING: No blkio throttle.read_iops_device support
WARNING: No blkio throttle.write_iops_device support
WARNING: daemon is not using the default seccomp profile

What happened?

I'm using TestContainer to test a .NET Core 6 API, that uses two database containers (Postgres and SQL Server). The API is a multi-tenanted application (database per tenant); the Postgres database stores a directory of the tenant databases (which are hosted in SQL Server). When I run my tests through TestContainer on my dev PC, the tests all pass. However, when I then run the same tests as part of my CI workflow in GitHub Actions, then the tests fail on set up with an ArgumentNullException when initialising the DbContext with the connection string retrieved from the test container.

I've attached a sample log from GitHub Actions, as well as the code for my TestContainer setup. The exception is being thrown during that setup, because directoryConnectionString is being set as null when calling _postgresContainer.GetConnectionString(). Would the reason for that be that Postgres hasn't finished initialising, or somesuch (although the logs suggest readiness checks have been done)?

Relevant log output

Test run for /home/runner/work/api/api/api/tests/MyProject.Api.Tests/bin/Release/net6.0/MyProject.Api.Tests.dll (.NETCoreApp,Version=v6.0)
Microsoft (R) Test Execution Command Line Tool Version 17.7.1 (x64)
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
[testcontainers.org 00:00:00.06] Connected to Docker:
  Host: unix:///var/run/docker.sock
  Server Version: 24.0.6
  Kernel Version: 6.2.0-1011-azure
  API Version: 1.43
  Operating System: Ubuntu 22.04.3 LTS
  Total Memory: 6.76 GB
[testcontainers.org 00:00:00.25] Searching Docker registry credential in Auths
[testcontainers.org 00:00:00.26] Docker registry credential https://index.docker.io/v1/ found
[testcontainers.org 00:00:01.06] Searching Docker registry credential in CredHelpers
[testcontainers.org 00:00:01.06] Searching Docker registry credential in CredsStore
[testcontainers.org 00:00:03.15] Docker image testcontainers/ryuk:0.5.1 created
[testcontainers.org 00:00:03.22] Docker container 5b553696ccd1 created
[testcontainers.org 00:00:03.27] Start Docker container 5b553696ccd1
[testcontainers.org 00:00:04.23] Wait for Docker container 5b553696ccd1 to complete readiness checks
[testcontainers.org 00:00:04.23] Docker container 5b553696ccd1 ready
[testcontainers.org 00:00:04.26] Searching Docker registry credential in Auths
[testcontainers.org 00:00:04.26] Searching Docker registry credential in Auths
[testcontainers.org 00:00:04.26] Searching Docker registry credential in CredHelpers
[testcontainers.org 00:00:04.26] Searching Docker registry credential in CredsStore
[testcontainers.org 00:00:04.26] Docker registry credential mcr.microsoft.com not found
[testcontainers.org 00:00:22.62] Docker image postgres:latest created
[testcontainers.org 00:00:22.65] Docker container 2882012ab637 created
[testcontainers.org 00:00:22.66] Start Docker container 2882012ab637
[testcontainers.org 00:00:23.04] Wait for Docker container 2882012ab637 to complete readiness checks
[testcontainers.org 00:00:25.10] Docker container 2882012ab637 ready
[testcontainers.org 00:00:48.17] Docker image mcr.microsoft.com/mssql/server:2022-latest created
[testcontainers.org 00:00:48.87] Docker container 45b62b9569bf created
[testcontainers.org 00:00:48.87] Start Docker container 45b62b9569bf
[testcontainers.org 00:00:49.15] Wait for Docker container 45b62b9569bf to complete readiness checks
[testcontainers.org 00:00:49.16] Execute "/opt/mssql-tools/bin/sqlcmd -Q SELECT 1;" at Docker container 45b62b9569bf
[testcontainers.org 00:00:55.05] Docker container 45b62b9569bf ready
[xUnit.net 00:00:57.02]     MyProject.Api.Tests.Application.Employees.GetPermissions.WhenGettingEmployeePermissionsAndEmployeeIsNotFound.ItShouldHaveTraceIdHeader [FAIL]
[xUnit.net 00:00:57.07]     MyProject.Api.Tests.Application.Employees.GetPermissions.WhenGettingEmployeePermissionsAndEmployeeIsNotFound.ItShouldReturnTheExpectedStatus [FAIL]
  Failed MyProject.Api.Tests.Application.Employees.GetPermissions.WhenGettingEmployeePermissionsAndEmployeeIsNotFound.ItShouldHaveTraceIdHeader [1 s]
  Error Message:
[xUnit.net 00:00:57.08]     MyProject.Api.Tests.Application.Employees.GetPermissions.WhenGettingEmployeePermissionsAndRequestIsOk.ItShouldHaveTraceIdHeader [FAIL]
   System.ArgumentNullException : Value cannot be null. (Parameter 'connectionString')
  Stack Trace:
     at System.ArgumentNullException.Throw(String paramName)
   at System.ArgumentNullException.ThrowIfNull(Object argument, String paramName)
   at MyProject.Shared.Database.Directory.ServiceCollectionExtensions.AddDirectoryDbContext(IServiceCollection services, String connectionString) in /home/runner/work/api/api/shared/src/MyProject.Shared.Database.Directory/ServiceCollectionExtensions.cs:line 18
   at Program.<Main>$(String[] args) in /home/runner/work/api/api/api/src/MyProject.Api.Application/Program.cs:line 127
--- End of stack trace from previous location ---
   at Microsoft.Extensions.Hosting.HostFactoryResolver.HostingListener.CreateHost()
   at Microsoft.Extensions.Hosting.HostFactoryResolver.<>c__DisplayClass10_0.<ResolveHostFactory>b__0(String[] args)
   at Microsoft.AspNetCore.Mvc.Testing.DeferredHostBuilder.Build()
   at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.CreateHost(IHostBuilder builder)
   at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.ConfigureHostBuilder(IHostBuilder hostBuilder)
   at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.EnsureServer()
   at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.CreateDefaultClient(DelegatingHandler[] handlers)
   at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.CreateDefaultClient(Uri baseAddress, DelegatingHandler[] handlers)
   at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.CreateClient(WebApplicationFactoryClientOptions options)
   at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.CreateClient()
   at IntegrationTest.CreateClient(String organisationId) in /home/runner/work/api/api/api/tests/MyProject.Api.Tests/Application/IntegrationTest.cs:line 34
   at MyProject.Api.Tests.Application.Employees.GetPermissions.WhenGettingEmployeePermissionsAndEmployeeIsNotFound.ItShouldHaveTraceIdHeader() in /home/runner/work/api/api/api/tests/MyProject.Api.Tests/Application/Employees/GetPermissions/WhenGettingEmployeePermissionsAndEmployeeIsNotFound.cs:line 20
--- End of stack trace from previous location ---

Additional information

TestContainers are initialised as follows:

public class TestApiApplication : WebApplicationFactory<Program>, IAsyncLifetime
{
    private readonly MsSqlContainer _sqlContainer = new MsSqlBuilder()
        .WithImage("mcr.microsoft.com/mssql/server:2022-latest")
        .WithPassword("Password01$")
        .Build();

    private readonly PostgreSqlContainer _postgresContainer = new PostgreSqlBuilder()
        .WithImage("postgres:latest")
        .WithUsername("admin")
        .WithPassword("Password01$")
        .WithDatabase("directory")
        .Build();

    Task IAsyncLifetime.InitializeAsync()
    {
        var startSqlServer = _sqlContainer.StartAsync();
        var startPostgres = _postgresContainer.StartAsync();
        return Task.WhenAll(startSqlServer, startPostgres);
    }

    Task IAsyncLifetime.DisposeAsync()
    {
        var stopSqlServer = _sqlContainer.StopAsync();
        var stopPostgres = _postgresContainer.StopAsync();
        return Task.WhenAll(stopSqlServer, stopPostgres);
    }

    protected override void ConfigureWebHost(IWebHostBuilder builder)
    {
        builder.ConfigureTestServices(services =>
        {
            ConfigureDirectoryDbContext(services);
        });
    }

    private void ConfigureDirectoryDbContext(IServiceCollection services)
    {
        var descriptorType = typeof(DbContextOptions<DirectoryDbContext>);
        var descriptor = services.SingleOrDefault(s => s.ServiceType == descriptorType);
        if (descriptor is not null)
        {
            services.Remove(descriptor);
        }

        var directoryConnectionString = _postgresContainer.GetConnectionString();
        services.AddDirectoryDbContext(directoryConnectionString);
    }
}

[HofmeisterAn]

I am unable to reproduce the issue, and I do not think GetConnectionString() can return null at all:

https://github.com/testcontainers/testcontainers-dotnet/blob/560b64fff57fb659cac3e6fc2570a034486d763b/src/Testcontainers.PostgreSql/PostgreSqlContainer.cs#L24-L33

Considering the stack trace, it looks like the ArgumentNullException is not thrown from the tests (ConfigureDirectoryDbContext(IServiceCollection)). It is thrown from your main application.

System.ArgumentNullException : Value cannot be null. (Parameter 'connectionString')
Stack Trace:
  at System.ArgumentNullException.Throw(String paramName)
at System.ArgumentNullException.ThrowIfNull(Object argument, String paramName)
at MyProject.Shared.Database.Directory.ServiceCollectionExtensions.AddDirectoryDbContext(IServiceCollection services, String connectionString) in /home/runner/work/api/api/shared/src/MyProject.Shared.Database.Directory/ServiceCollectionExtensions.cs:line 18
at Program.<Main>$(String[] args) in /home/runner/work/api/api/api/src/MyProject.Api.Application/Program.cs:line 127

I do not think this is a TC for .NET issue. Can you share a reproducer?

[davidkeaveny]

You're right - the TestApiApplication was verifying that a connection string had been set in configuration, and throwing an ArgumentNullException if no connection string was present. Seeing as this is the 21st century and nice developers don't store their connection strings in plaintext to expose in their GitHub repositories, the connection string was in user secrets on my dev PC, which are obviously not going to be present when running in GitHub Actions, hence the error before the test framework could do its bit and swap out the default implementation with the TestContainer-backed implementation.

So I modified the call to dotnet test in the GitHub Action to set a default connection string in an environment variable, now the tests Just Work ^TM.

Sorry to waste your time, and thank you for producing such a useful tool!