Closed razfriman closed 4 years ago
It seems like it is saying that membarrier
is not implemented yet - Which I see a related issue here: https://github.com/google/gvisor/issues/267
Hi, thanks for the report. I've got a few more questions.
Could you link to an example Docker image / Dockerfile that has this problem? You've described it, but being unfamiliar with .NET applications I'm not sure I'd create one correctly.
Also, what is the behavior you actually see? Are there additional relevant log messages? i.e., is the application crashing, failing to respond to requests, etc?
If possible, could you try running the container under gVisor locally (https://gvisor.dev/docs/user_guide/docker/) to see if the issue reproduces there?
Thanks!
Sure:
I have created a sample project that attempts to reproduce the issue with minimal amount of code.
GitHub Repository: https://github.com/razfriman/csharp-cloud-run
Hosted Docker Image: https://hub.docker.com/r/razfriman/cloud-run-web-app
Note: I created this sample app using the .NET CLI (dotnet new webapp
) and added a Dockerfile.
Then I created the Cloud Run
using the above docker image by adding it to my current projects GCR registry.
When I run the same container in GCE, everything works as expected. The application seems like it is running correctly, but gvisor is mentioning a message about membarrier
and it never marks the container as healthy
.
Logs:
2019-10-18T17:51:04.787Z Cloud Run CreateService csharp-cloud-run raz@gmail.com {"@type":"type.googleapis.com/google.cloud.audit.AuditLog","authenticationInfo":{"principalEmail":"raz@gmail.com"},"requestMetadata":{"callerIp":"111.220.68.158","callerSuppliedUserAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865… I Cloud Run CreateService csharp-cloud-run raz@gmail.com
2019-10-18T17:51:39.476936827Z Container Sandbox Limitation: Unsupported syscall membarrier(0x0,0x0,0x41,0x1,0x800000,0x1). Please, refer to https://gvisor.dev/c/linux/amd64/membarrier for more information. D
2019-10-18T17:51:42.952902Z [40m[1m[33mwarn[39m[22m[49m: Microsoft.AspNetCore.DataProtection.Repositories.FileSystemXmlRepository[60] A
2019-10-18T17:51:42.953213Z Storing keys in a directory '/home/.aspnet/DataProtection-Keys' that may not be persisted outside of the container. Protected data will be unavailable when container is destroyed. A
2019-10-18T17:51:43.142357Z [40m[1m[33mwarn[39m[22m[49m: Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager[35] A
2019-10-18T17:51:43.142387Z No XML encryptor configured. Key {4f315068-50c0-4063-a839-4d181277ece7} may be persisted to storage in unencrypted form. A
2019-10-18T17:51:43.615818Z [40m[32minfo[39m[22m[49m: Microsoft.Hosting.Lifetime[0] A
2019-10-18T17:51:43.615867Z Now listening on: http://[::]:80 A
2019-10-18T17:51:43.617115Z [40m[32minfo[39m[22m[49m: Microsoft.Hosting.Lifetime[0] A
2019-10-18T17:51:43.617146Z Application started. Press Ctrl+C to shut down. A
2019-10-18T17:51:43.617156Z [40m[32minfo[39m[22m[49m: Microsoft.Hosting.Lifetime[0] A
2019-10-18T17:51:43.617164Z Hosting environment: Production A
2019-10-18T17:51:43.617172Z [40m[32minfo[39m[22m[49m: Microsoft.Hosting.Lifetime[0] A
2019-10-18T17:51:43.617181Z Content root path: /app A
I can attempt to run gvisor
locally, but I do not have a machine with Linux so that may take some time
@prattmic - It turns out that Cloud Run needed some extra configuration that most .NET apps do not require.
The warning from gvisor
was a red herring as I still see it with the corrected configuration, however, the application starts up as expected.
Closing
I have run into an issue with gvisor while trying to deploy a containerized C# ASP.NET Core application to Google Cloud Run.
I have used the default .NET Core 3 templates and dockerized the application. It works locally via docker/docker-compose.
When attempting to deploy to Google Cloud Run, the application deploys and seems to start successfully according to the application logs, however, the application fails to report healthy and eventually fails the deployment.
Here is logs from the Google Log Viewer:
I have tried to compile the container with various flags and fail with the same issue everytime.
I assume this is a gvisor issue account to the error message. Do you have any suggestions?
TODO: Figure out how to get real debug logs.
For all other issues, please attach debug logs. To get debug logs, follow the instructions here: https://gvisor.dev/docs/user_guide/debugging/
Other useful information to include is:
runsc -v
docker version
ordocker info
if more relevantuname -a
-git describe