[tests] Hosting.Container tests failing with ` /app/static/cachebuster.txt: No such file or directory`

radical commented 2 months ago

Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=734641&view=ms.vss-test-web.build-test-results-tab Failed Aspire.Hosting.Containers.Tests.WithDockerfileTests.AddDockerfileLaunchesContainerSuccessfully [3 m 1 s] with:

  Error Message:
   System.TimeoutException : The operation has timed out.
  Stack Trace:
     at Aspire.Hosting.Containers.Tests.WithDockerfileTests.WaitForResourceAsync(DistributedApplication app, String resourceName, String resourceState, Nullable`1 timeout) in /_/tests/Aspire.Hosting.Containers.Tests/WithDockerfileTests.cs:line 819
   at Aspire.Hosting.Containers.Tests.WithDockerfileTests.AddDockerfileLaunchesContainerSuccessfully() in /_/tests/Aspire.Hosting.Containers.Tests/WithDockerfileTests.cs:line 105
--- End of stack trace from previous location ---
  Standard Output Messages:
 | [2024-07-09T17:33:37] Aspire.Hosting.DistributedApplication Information: Aspire version: 8.1.0-ci
 | [2024-07-09T17:33:37] Aspire.Hosting.DistributedApplication Information: Distributed application starting.
 | [2024-07-09T17:33:37] Aspire.Hosting.DistributedApplication Information: Application host directory is: /mnt/vss/_work/1/s/tests/Aspire.Hosting.Tests
 | [2024-07-09T17:33:37] Aspire.Hosting.Dcp.DcpHostService Information: Starting DCP with arguments: start-apiserver --monitor 41536 --detach --kubeconfig "/tmp/aspire.YdODAW/kubeconfig"
 | [2024-07-09T17:33:38] Aspire.Hosting.Dcp.start-apiserver.api-server Information: Starting API server...
 | [2024-07-09T17:33:38] Aspire.Hosting.Dcp.start-apiserver.api-server Information: API server started  {"Address": "127.0.0.1", "Port": 36673}
 | [2024-07-09T17:33:38] Aspire.Hosting.Dcp.start-apiserver.dcp-host Information: Starting DCP controller host
 | [2024-07-09T17:33:38] Aspire.Hosting.Dcp.start-apiserver.dcp-host Information: Started all services  {"count": 1}
 | [2024-07-09T17:33:38] Aspire.Hosting.Dcp.dcpctrl.IdeExecutableRunner Information: Executables cannot be started via IDE: missing required environment variable 'DEBUG_SESSION_PORT'
 | [2024-07-09T17:33:38] Aspire.Hosting.Dcp.dcpctrl Information: starting controller manager
 | [2024-07-09T17:33:38] Aspire.Hosting.DistributedApplication Information: Distributed application started. Press Ctrl+C to shut down.
 | [2024-07-09T17:33:39] Aspire.Hosting.Dcp.dcpctrl.ContainerReconciler Error: could not build the image    {"Container": {"name":"testcontainer-fgymmffm-3a7a1a22"}, "Reconciliation": 3, "error": "docker command 'BuildImage' returned with non-zero exit code 1: command output: Stdout: '' Stderr: '#1 [internal] load build definition from Dockerfile\n#1 sha256:543acee9a7b27570bd9f0aaa0097e248d68254ce72e0585e1fd10da2ca406fe0\n#1 transferring dockerfile: 428B done\n#1 DONE 0.0s\n\n#2 [internal] load .dockerignore\n#2 sha256:e1b191366a3815d8b74cf7b1014abb59ae1486f294a8815361feda120c014b8f\n#2 transferring context: 2B done\n#2 DONE 0.0s\n\n#3 [internal] load metadata for mcr.microsoft.com/k8se/quickstart:latest\n#3 sha256:7ca2d672e192e4c3fcffbc8346f5cb4c5b8c331c05951b57c29bce83d119286f\n#3 DONE 0.1s\n\n#4 [builder 1/3] FROM mcr.microsoft.com/k8se/quickstart:latest@sha256:08c960a417d963603b4f47686220a1f44438e27d41bcbcb5ffbe13bc3204a271\n#4 sha256:69aa65b13d5f77f6516aab5b55411a9975435d35b0956feee1e226727196c5b0\n#4 CACHED\n\n#5 [builder 2/3] RUN echo 92ea7f4b-6e99-47f6-b81e-41ba54d0c991 > /app/static/cachebuster.txt\n#5 sha256:aa467cb1ec2e0a6588e12440f0203e43753a4cfc891310dafd6c95c400fc8d9e\n#5 0.242 /bin/sh: line 1: /app/static/cachebuster.txt: No such file or directory\n#5 ERROR: executor failed running [/bin/sh -c echo 92ea7f4b-6e99-47f6-b81e-41ba54d0c991 > /app/static/cachebuster.txt]: exit code: 1\n------\n > [builder 2/3] RUN echo 92ea7f4b-6e99-47f6-b81e-41ba54d0c991 > /app/static/cachebuster.txt:\n------\nexecutor failed running [/bin/sh -c echo 92ea7f4b-6e99-47f6-b81e-41ba54d0c991 > /app/static/cachebuster.txt]: exit code: 1\n'"}

The dcp error in the log expands to:

[xUnit.net 00:03:22.70]         | [2024-07-09T17:30:41] Aspire.Hosting.Dcp.dcpctrl.ContainerReconciler
Error: could not build the image
{
   "Container" : {
      "name" : "testcontainer-huajnktk-19a4017e"
   },
   "Reconciliation" : 3,
   "error" : "docker command 'BuildImage' returned with non-zero exit code 1: command output: Stdout: '' Stderr: '#2 [internal] load .dockerignore
#2 sha256:0aeb500d378ffd97c6fd37cf3559174af18b913fec23f1ed0243cc18140aedab
#2 transferring context: 2B done
#2 DONE 0.0s

#1 [internal] load build definition from Dockerfile
#1 sha256:f9005bb19f66025452e591bcec1fac069071bf21ebad9dc2f8665465c0ae7702
#1 transferring dockerfile: 615B 0.0s done
#1 DONE 0.1s

#3 [internal] load metadata for mcr.microsoft.com/k8se/quickstart:latest
#3 sha256:7ca2d672e192e4c3fcffbc8346f5cb4c5b8c331c05951b57c29bce83d119286f
#3 DONE 0.2s

#4 [builder 1/3] FROM mcr.microsoft.com/k8se/quickstart:latest@sha256:08c960a417d963603b4f47686220a1f44438e27d41bcbcb5ffbe13bc3204a271
#4 sha256:69aa65b13d5f77f6516aab5b55411a9975435d35b0956feee1e226727196c5b0
#4 resolve mcr.microsoft.com/k8se/quickstart:latest@sha256:08c960a417d963603b4f47686220a1f44438e27d41bcbcb5ffbe13bc3204a271 done
#4 sha256:573aa0337010c6e8b107a0f7442a29dae66c2367d9f98647af2749c296c2c8fc 3.11kB / 3.11kB 0.2s done
#4 sha256:08c960a417d963603b4f47686220a1f44438e27d41bcbcb5ffbe13bc3204a271 1.36kB / 1.36kB done
#4 sha256:fec89a3bf5f700ce5111fba2afc3eb2f2bb217071e5be37d2ac97c7588b6485f 3.16kB / 3.16kB done
#4 sha256:24c026dce085c078ccb99910ab23372a306038e70d7fd27222cea5ae333f1973 10.49MB / 28.75MB 0.2s
#4 sha256:9b0ba995d6d394be901b0c7c5e0a5ca14327aa488ef6de617a809b3a952a75fb 4.46kB / 4.46kB 0.1s done
#4 sha256:3128bd9f33b6c43c5c6e58f623f1a83d20b9d17a7807eeba4bf06f93f6a9e822 4.19MB / 5.32MB 0.2s
#4 sha256:3554b4dd717c6b2cb3ec095cde0ea8decd3e6d923a705aa001e2dac143e6ae66 0B / 2.31kB 0.2s
#4 sha256:24c026dce085c078ccb99910ab23372a306038e70d7fd27222cea5ae333f1973 26.21MB / 28.75MB 0.3s
#4 sha256:3128bd9f33b6c43c5c6e58f623f1a83d20b9d17a7807eeba4bf06f93f6a9e822 5.32MB / 5.32MB 0.2s done
#4 sha256:3554b4dd717c6b2cb3ec095cde0ea8decd3e6d923a705aa001e2dac143e6ae66 2.31kB / 2.31kB 0.3s done
#4 sha256:24c026dce085c078ccb99910ab23372a306038e70d7fd27222cea5ae333f1973 28.75MB / 28.75MB 0.3s done
#4 extracting sha256:24c026dce085c078ccb99910ab23372a306038e70d7fd27222cea5ae333f1973
#4 extracting sha256:24c026dce085c078ccb99910ab23372a306038e70d7fd27222cea5ae333f1973 1.3s done
#4 extracting sha256:9b0ba995d6d394be901b0c7c5e0a5ca14327aa488ef6de617a809b3a952a75fb done
#4 extracting sha256:3128bd9f33b6c43c5c6e58f623f1a83d20b9d17a7807eeba4bf06f93f6a9e822
#4 extracting sha256:3128bd9f33b6c43c5c6e58f623f1a83d20b9d17a7807eeba4bf06f93f6a9e822 0.3s done
#4 extracting sha256:573aa0337010c6e8b107a0f7442a29dae66c2367d9f98647af2749c296c2c8fc done
#4 extracting sha256:3554b4dd717c6b2cb3ec095cde0ea8decd3e6d923a705aa001e2dac143e6ae66 done
#4 DONE 2.7s

#5 [builder 2/3] RUN echo afa782d8-cc52-4c3d-8bee-79861e3aa4ea > /app/static/cachebuster.txt
#5 sha256:117128d7f9a8edbbc7232f486028ad612ba44bd3abb2e80935b15d0e02c3882b
#5 0.566 /bin/sh: line 1: /app/static/cachebuster.txt: No such file or directory
#5 ERROR: executor failed running [/bin/sh -c echo afa782d8-cc52-4c3d-8bee-79861e3aa4ea > /app/static/cachebuster.txt]: exit code: 1
------
 > [builder 2/3] RUN echo afa782d8-cc52-4c3d-8bee-79861e3aa4ea > /app/static/cachebuster.txt:
------
executor failed running [/bin/sh -c echo afa782d8-cc52-4c3d-8bee-79861e3aa4ea > /app/static/cachebuster.txt]: exit code: 1
'"
}

this is being hit consistently since https://dev.azure.com/dnceng-public/public/_build/results?buildId=734641&view=results (unrelated changes though)
I can reproduce this locally. The /app path does not exist, and that's where RUN echo afa782d8-cc52-4c3d-8bee-79861e3aa4ea > /app/static/cachebuster.txt fails.

Aspire.Hosting.Containers.Tests.WithDockerfileTests.AddDockerfileLaunchesContainerSuccessfully, and Aspire.Hosting.Containers.Tests.WithDockerfileTests.WithDockerfileWithParameterLaunchesContainerSuccessfully also fails the same way.

cc @mitchdenny @eerhardt @karolz-ms

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "",
  "ErrorPattern": "/bin/sh: line 1: /app/static/cachebuster.txt: No such file or directory",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Known issue validation

Build: :mag_right: https://dev.azure.com/dnceng-public/public/_build/results?buildId=734641 Error message validated: [/bin/sh: line 1: /app/static/cachebuster.txt: No such file or directory] Result validation: :x: Known issue did not match with the provided build. Validation performed at: 7/9/2024 9:48:59 PM UTC

Report

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	0

eerhardt commented 2 months ago

It looks like latest in

https://github.com/dotnet/aspire/blob/f647a661944e1439fcc15cf5e25ac6812d713002/tests/Aspire.Hosting.Containers.Tests/WithDockerfileTests.cs#L825

got updated and the docker image we are pointing to no longer has a /app directory. Which means when the command is trying to write a file to /app/static the directory doesn't exist and an error occurs.

I'm not sure what the old version of mcr.microsoft.com/k8se/quickstart looked like. From what I could tell, the latest version is located at https://github.com/microsoft/k4apps/tree/main/samples/apps/quickstart (maybe?). At least when I pulled the current mcr.microsoft.com/k8se/quickstart:latest, that's what app it looked like got pulled.

We are disabling these tests for now until a correct fix can be made for them. cc @mitchdenny

radical commented 2 months ago

Fixed by https://github.com/dotnet/aspire/pull/4832 .

dotnet / aspire