search

microsoft / service-fabric-issues

This repo is for the reporting of issues found with Azure Service Fabric.
168 stars 21 forks source link

Can't debug vanilla ASP.NET Core application locally when using containers in WS2019 with VS2019 #1518

Closed esardaya closed 5 years ago

esardaya commented 5 years ago

I've been trying for quite a while to debug a simple ASP.NET core project within a container deployed as part of a single-service single-node SF application without any success.

Repro Steps

  1. Create a new ASP.NET Core Web Application using the project creation dialog in Visual Studio 2019 (in my case with the API template) without enabling HTTPS nor Docker support.
  2. Add Container Orchestrator Support for Service Fabric (note the created API project does not have any manual modifications at this point).
  3. Update the generated Dockerfile to use microsoft/dotnet:2.1-aspnetcore-runtime-nanoserver-1809 as the base image. (The default image doesn't seem to be compatible with WS2019)
  4. Try to run the Service Fabric Application in debug mode (set as startup project and F5)

Expected: Application is deployed successfully and debugger is attached.
Actual: Container fails to start with the following error (from Event Viewer in host): Failed to start Container. ContainerName=sf-0-fd8822b0-8f13-46de-8094-3bece876b2a0_26e0bd44-f8a0-4940-a44c-1e2a061f360a, ApplicationId=WebToSFContainerApplicationType_App0, ApplicationName=fabric:/WebToSFContainerApplication. DockerRequest returned StatusCode=InternalServerError with ResponseBody={"message":"container b1625a819fc8c7a89e287e16aa715b8c999316bea21dd0f16dd4be246e28b21f encountered an error during CreateProcess: failure in a Windows system call: Access is denied. (0x5) extra info: {\"CommandLine\":\"C:\\\\remote_debugger\\\\x64\\\\msvsmon.exe /noauth /anyuser /silent /nostatus /noclrwarn /nosecuritywarn /nofirewallwarn /nowowwarn /timeout:2147483646\",\"User\":\"ContainerUser\",\"WorkingDirectory\":\"C:\\\\app\",\"Environment\":{\"ASPNETCORE_ENVIRONMENT\":\"Development\",\"ASPNETCORE_URLS\":\"http://+:80\",\"DOTNET_RUNNING_IN_CONTAINER\":\"true\",\"DOTNET_RUNNING_IN_CONTAINERS\":\"true\",\"DOTNET_USE_POLLING_FILE_WATCHER\":\"1\",\"FabricCodePath\":\"C:\\\\SFFabricBin\",\"FabricLogRoot\":\"C:\\\\SFFabricLog\",\"FabricPackageFileName\":\"C:\\\\SFPackageRoot\\\\Fabric.Package.current.xml\",\"Fabric_ApplicationHostId\":\"fd8822b0-8f13-46de-8094-3bece876b2a0\",\"Fabric_ApplicationHostType\":\"Activated_SingleCodePackage\",\"Fabric_ApplicationId\":\"WebToSFContainerApplicationType_App0\",\"Fabric_ApplicationName\":\"fabric:/WebToSFContainerApplication\",\"Fabric_CodePackageInstanceSeqNum\":\"132025169178835176\",\"Fabric_CodePackageName\":\"Code\",\"Fabric_ContainerName\":\"sf-0-fd8822b0-8f13-46de-8094-3bece876b2a0_26e0bd44-f8a0-4940-a44c-1e2a061f360a\",\"Fabric_Endpoint_IPOrFQDN_WebToSFContainerTypeEndpoint\":\"<redacted>\",\"Fabric_Endpoint_WebToSFContainerTypeEndpoint\":\"9005\",\"Fabric_Folder_App_Log\":\"C:\\\\SFApplications\\\\WebToSFContainerApplicationType_App0\\\\log\",\"Fabric_Folder_App_Temp\":\"C:\\\\SFApplications\\\\WebToSFContainerApplicationType_App0\\\\temp\",\"Fabric_Folder_App_Work\":\"C:\\\\SFApplications\\\\WebToSFContainerApplicationType_App0\\\\work\",\"Fabric_Folder_Application\":\"C:\\\\SFApplications\\\\WebToSFContainerApplicationType_App0\",\"Fabric_Folder_Application_OnHost\":\"F:\\\\SfDevCluster\\\\Data\\\\_App\\\\_Node_0\\\\WebToSFContainerApplicationType_App0\",\"Fabric_IsCodePackageActivatorHost\":\"false\",\"Fabric_IsContainerHost\":\"true\",\"Fabric_NET-0-[Other]\":\"Other\",\"Fabric_NetworkingMode\":\"Other\",\"Fabric_NodeIPOrFQDN\":\"<redacted>\",\"Fabric_NodeId\":\"bf865279ba277deb864a976fbf4c200e\",\"Fabric_NodeName\":\"_Node_0\",\"Fabric_PartitionId\":\"1e575f1f-0bce-40eb-8323-09dff21d8c98\",\"Fabric_RuntimeConnectionAddress\":\"<redacted>:56260\",\"Fabric_RuntimeSslConnectionAddress\":\"<redacted>:56260\",\"Fabric_RuntimeSslConnectionCertEncodedBytes\":\"MIIC5jCCAc6gAwIBAgIQVVMfn634a6xOrPXuPRyBOTANBgkqhkiG9w0BAQUFADAvMS0wKwYDVQQDEyQ5NmMwMmI3OS0yZjNiLTRiZmUtOTc1YS1hYTAyMzVkMDhjYzgwHhcNMTkwNTE1MjE1MDQzWhcNMjkwNTEzMjE1MDQzWjAvMS0wKwYDVQQDEyQ5NmMwMmI3OS0yZjNiLTRiZmUtOTc1YS1hYTAyMzVkMDhjYzgwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDBwxPj/aybz7r38V488rQ/lKZ48QgCT9QIst8iSEn6xf42SurlQ7DKFMeIQqQ0WbMUxbteE06NF0jpzFy6xnZ61JpD/Owzo54T5VO7D4tzB5j6S3eNZC0azGS6Hxp1eC8Ot06PJWDqsgpMIv8I4Knbn6lwbThnKMJYr6YEifAlUF640Fkk/M1jE7UxJzfBZV/I445N7BLt57WSO9G1cF/p4TVSrVTQEmAfGomSnbXmqwmubZ3dWYrnVUrYklbotW+quk2jPtNK6MY+xdXOzviwiEjx8/mj9eGxlJk9IQrUiJp2giu5/J2VJbMhvxOCNBkROUpgfi/96r7lRYwPTI79AgMBAAEwDQYJKoZIhvcNAQEFBQADggEBAAh6Ino0Y+EQIA6DtoyPHlWwbDrW4YrApAKItbZX0KWSKTwY2aXuo+iqn5t6gLDOTw2joniQ9qnCFhSd3aH2uNW64uiG9tuzUFeZ92sp1H9/AAiV7/s8/X3wEy0un1QAWS049djDDWMWCzU8s2NKrMcKz0QLVq2LFjyhkxWoFOFbGVeZspOOtpAih5Y27QIo6K+68YWC7wXoxzxl2zs0LrNXAnf2dW4WnH/WVU6vKeL/EUKwSZSUCeWtb7xMln1IFfiP4deZlNuqmDs3fXl6WrXHQVcxc/vTL9BsRw1g1V9957NrSsc0fiJzPsGDnNAkhICoClrJ+qjP6FEs2tEIQM4=\",\"Fabric_RuntimeSslConnectionCertKey\":\"BwIAAACkAABSU0EyAAgAAAEAAQD9jkwPjEXlvur9L35gSjkRGTSCE78hsyWVnfy5K4J2mojUCiE9mZSx4fWj+fPxSIiw+M7O1cU+xuhK0z6jTbqqb7XoVpLYSlXnilndnW2uCavmtZ2SiRofYBLQVK1SNeHpX3C10TuSteftEuxNjuPIX2XBNycxtRNjzfwkWdC4XlAl8IkEpq9YwihnOG1wqZ/bqeAI/yJMCrLqYCWPTrcOL3h1Gh+6ZMwaLWSNd0v6mAdziw+7U+UTnqMz7PxDmtR6dsa6XMzpSBeNThNeu8UUs1k0pEKIxxTKsEPl6ko2/sX6SUgi37II1E8CCPF4ppQ/tPI8XvH3us+brP3jE8PBI1Odl0f8/HD+owZ9OORwIk0Ol+1IPjOqpj4WUttxZMwiaDmJ0lyte6P+5P41RjaRa2ClLGjPe7GkvDCO55SfZCAYMpokuu3rlYQDgamGoT75UFvqgEmtLmjU54UgIMvUXpas9cSMu60XfidRvvenrAY3dF09zyEwyQIbDhnReutfR3RqaPr+vtXTbiIxps4zRxXXN7kbbBfNORU1WVco4d2vN24YBcbLWvOiSHupMu7jrHpbo7rsnS8L2z85qaB8H8MgZgwUlRc+OUTpCFmvEXjDu/VcfHX1n6B8ZYV5UlAdNzW/pgHyreHHbAY3JOWQEM1Gh9ybKIe3IOENmZql0p0PNjyNkeffJ0ewR7wlr1zjTGWtvMFPr4rDnvuhj254o0Z6vJeAEgSOSRFO6bmIFmb7jngvGz2QO6CoRvFJFQ0U1UyActil2rtG811WmKjPiR998gwSsth1YWJLhw23PtdxkVZ8xesbtNUhe/jBijgPRPLEuB//W6JPCC8SNMQvLYBA4JxWg3E8ajAxyXjirj+iv1gBYmwSomZ/uP8L/iwirC/D+WXk2Gt1sodo7SjwjgvsT+H/DSaj1FYQtFhtl1zoUV4EsMgFlWogeU3bvgEGC0cExVujywbp4Bxaqtp0EZnegPzpd6g+lbsRgl8enkM0cbMpIyFQLNMdJjuLi78oYpurEqIkSAM2291PV92B/JE5voYbd1iPtJ5JA9JddkkKTMWErz1OhcedNo5wuz6MMRcKQrMjpFdK9WZgqOZ9ya3sGwIvYF+Zy2w7iKD5N9pHwvCbaXRMx9YTIkXBtKA5KZqEbT6NNJc7tf4VsNbOLSOv6FnXEV5eQm9hgmKMps2EUb4/KA/nyuhJ1IkpiDMfygABITIQsSV3MdR1WibyzvGYo/+Z4tcU9t1tEC99+W+oU1CFO8KqscZa5cZ5QLnEAGS3CUaHeS/h71wYY3gkaVtu5AlWIT63ddlVE0wnHUo+fRA7wjhS9wI1BKiwjoy4j4kaRSUWrU5CPOuPt53VS+ZVmEJzx5ALwg3awMLMP6ksPlP1EqTdg4+utK8cLWOBO/MPCliLAw4dCCJUHHMPQ93fM9mhS/tYz6SWwsI+6IFajEf46L+ZC5KzdWIxJmHk2f9Cr2AQIsYg5ac6R88qkHg4vu4SJ4+lls8F+rG8FZCxa1leYZkK/Dh1dmh0/GQ=\",\"Fabric_RuntimeSslConnectionCertThumbprint\":\"b7f5c9679fa02e27025fd07658ecb1d992bf6d8f\",\"Fabric_ServiceName\":\"fabric:/WebToSFContainerApplication/WebToSFContainer\",\"Fabric_ServicePackageActivationGuid\":\"1e575f1f-0bce-40eb-8323-09dff21d8c98\",\"Fabric_ServicePackageActivationId\":\"26e0bd44-f8a0-4940-a44c-1e2a061f360a\",\"Fabric_ServicePackageInstanceSeqNum\":\"132025169178835172\",\"Fabric_ServicePackageName\":\"WebToSFContainerPkg\",\"Fabric_ServicePackageVersionInstance\":\"1.0:1.0:132025170198138403\",\"NUGET_PACKAGES\":\"C:\\\\.nuget\\\\packages\"},\"CreateStdInPipe\":true,\"CreateStdOutPipe\":true,\"CreateStdErrPipe\":true,\"ConsoleSize\":[0,0]}"}

Note that if I publish the Service Fabric application locally without debugging (right-click Service Fabric application project -> Publish -> Local.1.Node.xml) then there aren't any issues and I can access the default 'values' (http://localhost:<port>/api/values) path just fine. This problem only occurs when trying to start the application in debug mode.

The only 2 changes I have made to my system that differ from a vanilla install (aside from installing VS, VSCode, etc.) are:

  1. Create my local Service Fabric cluster with 1 node and a different Data and Log location (custom data root and log root when running DevClusterSetup.ps1)
  2. Create a daemon.json file in the docker config folder with a data-root property so that images, etc. will be stored in a different location as seen here: https://docs.microsoft.com/en-us/virtualization/windowscontainers/manage-docker/configure-docker-daemon

I made those changes because my system drive is not very large and I wanted the aforementioned related files to be stored in my larger data drive.

Some additional data about my system: Windows version: Windows Server 2019 Version 1809 (17763.194) Visual Studio version: 16.0.4
Service Fabric SDK: 3.3.658.9590
Service Fabric Runtime: 6.4.658.9590 Docker info output: 

Containers: 1
 Running: 0
 Paused: 0
 Stopped: 1
Images: 21
Server Version: 18.09.0
Storage Driver: windowsfilter
 Windows:
Logging Driver: json-file
Plugins:
 Volume: local
 Network: ics l2bridge l2tunnel nat null overlay transparent
 Log: awslogs etwlogs fluentd gelf json-file local logentries splunk syslog
Swarm: inactive
Default Isolation: process
Kernel Version: 10.0 17763 (17763.1.amd64fre.rs5_release.180914-1434)
Operating System: Windows Server 2019 Datacenter Version 1809 (OS Build 17763.194)
OSType: windows
Architecture: x86_64
CPUs: 4
Total Memory: 28GiB
Name: <redacted>
ID: NPCY:7WU7:3JAS:J3EB:BTEF:MXVP:JZQ5:R3QV:3JQ4:V6UP:V6RT:IGZY
Docker Root Dir: F:\Docker
Debug Mode (client): false
Debug Mode (server): true
 File Descriptors: -1
 Goroutines: 25
 System Time: 2019-05-17T00:14:29.1495912+02:00
 EventsListeners: 1
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

WARNING: API is accessible on http://localhost:2375 without encryption.
         Access to the remote API is equivalent to root access on the host. Refer
         to the 'Docker daemon attack surface' section in the documentation for
         more information: https://docs.docker.com/engine/security/security/#docker-daemon-attack-surface
esardaya commented 5 years ago

I was able to repro this without Service Fabric by just creating an ASP.NET Core project with Docker support enabled and trying to debug it when using process isolation. Closing this issue since it doesn't seem to be specific to Service Fabric.