search

testcontainers / testcontainers-dotnet

A library to support tests with throwaway instances of Docker containers for all compatible .NET Standard versions.
https://dotnet.testcontainers.org
MIT License
3.74k stars 266 forks source link

[Bug]: CosmosDbContainer StartAsync Consistently Takes Over 100 Seconds to Complete #1241

Open SilverioMiranda opened 2 weeks ago

SilverioMiranda commented 2 weeks ago

Testcontainers version

3.9.0

Using the latest Testcontainers version?

Yes

Host OS

Windows

Host arch

x64

.NET version

8.0.304

Docker version

Client:
 Version:           27.1.1
 API version:       1.46
 Go version:        go1.21.12
 Git commit:        6312585
 Built:             Tue Jul 23 19:57:57 2024
 OS/Arch:           windows/amd64
 Context:           desktop-linux

Server: Docker Desktop 4.33.1 (161083)
 Engine:
  Version:          27.1.1
  API version:      1.46 (minimum version 1.24)
  Go version:       go1.21.12
  Git commit:       cc13f95
  Built:            Tue Jul 23 19:57:19 2024
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.7.19
  GitCommit:        2bf793ef6dc9a18e00cb12efb64355c2c9d5eb41
 runc:
  Version:          1.7.19
  GitCommit:        v1.1.13-0-g58aa920
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Docker info

Client:
 Version:    27.1.1
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.16.1-desktop.1
    Path:     C:\Program Files\Docker\cli-plugins\docker-buildx.exe
  compose: Docker Compose (Docker Inc.)
    Version:  v2.29.1-desktop.1
    Path:     C:\Program Files\Docker\cli-plugins\docker-compose.exe
  debug: Get a shell into any image or container (Docker Inc.)
    Version:  0.0.34
    Path:     C:\Program Files\Docker\cli-plugins\docker-debug.exe
  desktop: Docker Desktop commands (Alpha) (Docker Inc.)
    Version:  v0.0.14
    Path:     C:\Program Files\Docker\cli-plugins\docker-desktop.exe
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.2
    Path:     C:\Program Files\Docker\cli-plugins\docker-dev.exe
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.25
    Path:     C:\Program Files\Docker\cli-plugins\docker-extension.exe
  feedback: Provide feedback, right in your terminal! (Docker Inc.)
    Version:  v1.0.5
    Path:     C:\Program Files\Docker\cli-plugins\docker-feedback.exe
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.3.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-init.exe
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-sbom.exe
  scout: Docker Scout (Docker Inc.)
    Version:  v1.11.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-scout.exe

Server:
 Containers: 4
  Running: 3
  Paused: 0
  Stopped: 1
 Images: 10
 Server Version: 27.1.1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 2bf793ef6dc9a18e00cb12efb64355c2c9d5eb41
 runc version: v1.1.13-0-g58aa920
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
 Kernel Version: 5.15.153.1-microsoft-standard-WSL2
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 24
 Total Memory: 62.68GiB
 Name: docker-desktop
 ID: e4c6788e-aab3-4d57-aff3-f43a9db01344
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Labels:
  com.docker.desktop.address=npipe://\\.\pipe\docker_cli
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No blkio throttle.read_bps_device support
WARNING: No blkio throttle.write_bps_device support
WARNING: No blkio throttle.read_iops_device support
WARNING: No blkio throttle.write_iops_device support
WARNING: daemon is not using the default seccomp profile

What happened?

The call await _cosmosDbContainer.StartAsync(); consistently takes approximately 105,294 ms to complete.

This issue likely wasn't detected earlier because the test at CosmosDbContainerTest.cs#L18 is skipped during tests.

Here is the repository with the example experiencing this problem: https://github.com/SilverioMiranda/testcontainers-cosmosdb.

image image

Relevant log output

No response

Additional information

https://github.com/SilverioMiranda/testcontainers-cosmosdb

HofmeisterAn commented 2 weeks ago

Hi, why do you think this is an issue? Why shouldn't it take that long for the port and service to be up and running? I think it might even take a bit longer until Cosmos DB is fully ready. The following test takes just as long:

docker network create nw0
# Image size is ~2.35 GB
# docker pull mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:latest
# Image size is ~8 MB
# docker pull alpine:latest
$startTime = (Get-Date)
docker run --detach --network nw0 --net-alias cosmosdb --name cosmosdb --rm mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:latest
docker run --rm --network nw0 alpine sh -c 'while ! nc -z cosmosdb 8081; do echo "Retrying..."; sleep 1; done'
$timeTaken = (Get-Date) - $startTime
Write-Output "Connection to Cosmos DB established in $($timeTaken.TotalSeconds) seconds."

a8c88f724c0340f59f2e58a648a9c5a53493a7acefe5c226c053711bb11b5849
711b4ba876eb576085e55f0b59e8dc21a6592ac4107b5a806fcb7de9b2d79709
Retrying...
Retrying...
Connection to Cosmos DB established in 103.8691404 seconds.

I don't believe this is a problem with Testcontainers. It seems more like an "issue" with the Cosmos DB Emulator.

SilverioMiranda commented 2 weeks ago

It should be faster, right? When using it together with Microsoft Aspire running the emulator, it takes about 30 seconds to start and become available.

HofmeisterAn commented 1 week ago

Could you please share a reproducer (in Aspire) that runs the emulator locally and attempts to read the account information, similar to what we're doing in our test? This would make it easier to compare.

I remember that we changed the wait strategy because the previous one indicated readiness too early and was not reliable: https://github.com/testcontainers/testcontainers-dotnet/pull/1109. The CLI example takes just as long. I'm curious what Aspire uses.