dotnet / aspire

An opinionated, cloud ready stack for building observable, production ready, distributed applications in .NET
https://learn.microsoft.com/dotnet/aspire
MIT License
3.77k stars 438 forks source link

Container Networking not working with Rancher Desktop #6002

Open afscrome opened 2 weeks ago

afscrome commented 2 weeks ago

Is there an existing issue for this?

Describe the bug

Container networking doesn't seem to work with Rancher Desktop with 9.0.0-preview.4.24477.2. Works fine with 8.2.

Expected Behavior

I should be able to open the nginx homepage.

Steps To Reproduce

  1. Install Rancher Desktop - https://rancherdesktop.io/
  2. Run the following app host
    builder.AddContainer("nginx", "nginx")
    .WithHttpEndpoint(targetPort: 80);
  3. Try to visit the nginx endpoint

This fails with rancher, but works fine with docker desktop.

Exceptions (if any)

Going to the http endpoint url results in ERR_EMPTY_RESPONSE image

Going to the http target port url results in ERR_CONNECTION_RESET image

image

.NET Version info

.NET SDK:
 Version:           9.0.100-rc.1.24452.12
 Commit:            81a714c6d3
 Workload version:  9.0.100-manifests.c7c7028a
 MSBuild version:   17.12.0-preview-24422-09+d17ec720d

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.22631
 OS Platform: Windows
 RID:         win-x64
 Base Path:   C:\Program Files\dotnet\sdk\9.0.100-rc.1.24452.12\

.NET workloads installed:
Configured to use loose manifests when installing new manifests.
 [aspire]
   Installation Source: VS 17.11.35312.102
   Manifest Version:    9.0.0-preview.4.24454.4/8.0.100
   Manifest Path:       C:\Program Files\dotnet\sdk-manifests\8.0.100\microsoft.net.sdk.aspire\9.0.0-preview.4.24454.4\WorkloadManifest.json
   Install Type:        FileBased

Host:
  Version:      9.0.0-rc.1.24431.7
  Architecture: x64
  Commit:       static

.NET SDKs installed:
  8.0.400 [C:\Program Files\dotnet\sdk]
  8.0.401 [C:\Program Files\dotnet\sdk]
  9.0.100-rc.1.24452.12 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 8.0.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 9.0.0-rc.1.24452.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 8.0.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 9.0.0-rc.1.24431.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 8.0.8 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 9.0.0-rc.1.24452.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Other architectures found:
  x86   [C:\Program Files (x86)\dotnet]
    registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x86\InstallLocation]

Environment variables:
  Not set

global.json file:
  Not found

Learn more:
  https://aka.ms/dotnet/info

Download .NET:
  https://aka.ms/dotnet/download

Anything else?

  <Sdk Name="Aspire.AppHost.Sdk" Version="9.0.0-preview.4.24477.2" />
    <PackageReference Include="Aspire.Hosting.AppHost" Version="9.0.0-preview.4.24477.5" />
    <PackageReference Include="Aspire.Hosting.PostgreSQL" Version="9.0.0-preview.4.24477.5" />
    <PackageReference Include="Aspire.Hosting.Redis" Version="9.0.0-preview.4.24477.5" />
    <PackageReference Include="Aspire.Hosting.SqlServer" Version="9.0.0-preview.4.24477.5" />

dcp path: C:\Users\alex.nuget\packages\aspire.hosting.orchestration.win-x64\9.0.0-preview.4.24477.5\tools\dcp.exe

afscrome commented 16 hours ago

Still happens on RC1.

Interestingly, adding .WithLifetime(ContainerLifetime.Persistent) to containers makes them run fine in rancher.

afscrome commented 4 hours ago

I'll do some more investigation tonight, but currently it looks like container to container networking works (need to verify this), but host to container networking is broken. (I need to do some more investigation tonight)

Looks like the reason why rancher works with persistent containers, but not transient is the addition of the bridge network Image

var builder = DistributedApplication.CreateBuilder(args);

builder.AddContainer("nginx", "nginx")
    .WithHttpEndpoint(targetPort: 80)
    .WithHttpHealthCheck("/")
    ;

builder.AddContainer("nginx-persist", "nginx")
    .WithHttpEndpoint(targetPort: 80)
    .WithLifetime(ContainerLifetime.Persistent)
    .WithHttpHealthCheck("/")
    ;

await builder.Build().RunAsync();

Broken

        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "f39742e7dc5b71703e6e8c755dc87b019b2b0d724b16129871e8b5b7b7093b7a",
            "SandboxKey": "/var/run/docker/netns/f39742e7dc5b",
            "Ports": {
                "80/tcp": [
                    {
                        "HostIp": "127.0.0.1",
                        "HostPort": "32770"
                    }
                ]
            },
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {
                "default-aspire-network-g3grsv76o4": {
                    "IPAMConfig": {},
                    "Links": null,
                    "Aliases": [
                        "nginx"
                    ],
                    "MacAddress": "02:42:ac:13:00:03",
                    "NetworkID": "d4152830aea500aaffcc92291d567f19925fd61515e34019ca25890b814b1d93",
                    "EndpointID": "924c4d6bbef9b23d4cc0529252a8366f95364422bde6c8aa2e581ff96b4034de",
                    "Gateway": "172.19.0.1",
                    "IPAddress": "172.19.0.3",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "DriverOpts": {},
                    "DNSNames": [
                        "nginx-mzcehtdk",
                        "nginx",
                        "5ae99468a2f9"
                    ]
                }
            }
        }

Working

        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "26d75ec9690f957e1be3fee09661d13af1a32a215617e4b4a072855b01d2007d",
            "SandboxKey": "/var/run/docker/netns/26d75ec9690f",
            "Ports": {
                "80/tcp": [
                    {
                        "HostIp": "127.0.0.1",
                        "HostPort": "32768"
                    }
                ]
            },
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "c8f2d60def66cabdb6b796f8f849d3bbddc6476c2b86ebf84e6d28b9e94f37c4",
            "Gateway": "172.17.0.1",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "172.17.0.2",
            "IPPrefixLen": 16,
            "IPv6Gateway": "",
            "MacAddress": "02:42:ac:11:00:02",
            "Networks": {
                "bridge": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": null,
                    "MacAddress": "02:42:ac:11:00:02",
                    "NetworkID": "953836ba6cf5797ea9e634acf99f0a19a59f392ea519bb144b3bedcc31d888c9",
                    "EndpointID": "c8f2d60def66cabdb6b796f8f849d3bbddc6476c2b86ebf84e6d28b9e94f37c4",
                    "Gateway": "172.17.0.1",
                    "IPAddress": "172.17.0.2",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "DriverOpts": null,
                    "DNSNames": null
                },
                "default-aspire-network-g3grsv76o4": {
                    "IPAMConfig": {},
                    "Links": null,
                    "Aliases": [
                        "nginx-persist"
                    ],
                    "MacAddress": "02:42:ac:13:00:02",
                    "NetworkID": "d4152830aea500aaffcc92291d567f19925fd61515e34019ca25890b814b1d93",
                    "EndpointID": "1f0913d046b0da698edfa294dbbdfb35bd7c08a8a091a25ae41191aaeca36f64",
                    "Gateway": "172.19.0.1",
                    "IPAddress": "172.19.0.2",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "DriverOpts": {},
                    "DNSNames": [
                        "nginx-persist-5eea5367",
                        "nginx-persist",
                        "642dc353be59"
                    ]
                }
            }
        }
davidfowl commented 4 hours ago

cc @danegsta for ideas