search

dotnet / aspire

Tools, templates, and packages to accelerate building observable, production-ready apps
https://learn.microsoft.com/dotnet/aspire
MIT License
3.93k stars 480 forks source link

Odd errors if `Aspire.Dashboard.exe` gets orphaned from a previous run #5546

Open afscrome opened 2 months ago

afscrome commented 2 months ago

Is there an existing issue for this?

Describe the bug

Not entirely sure how I've managed this, but I somehow ended up with an orphaned Aspire.Dashboard.exe process. Once this process was orphaned, all further attempts to start my aspire projects failed with the below logs. 

fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.657Z Unhandled exception. System.IO.IOException: Failed to bind to address https://127.0.0.1:17054: address already in use.
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z  ---> Microsoft.AspNetCore.Connections.AddressInUseException: Only one usage of each socket address (protocol/network address/port) is normally permitted.
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z  ---> System.Net.Sockets.SocketException (10048): Only one usage of each socket address (protocol/network address/port) is normally permitted.
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at System.Net.Sockets.Socket.UpdateStatusAfterSocketErrorAndThrowException(SocketError error, Boolean disconnectOnFailure, String callerName)
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at System.Net.Sockets.Socket.DoBind(EndPoint endPointSnapshot, SocketAddress socketAddress)
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at System.Net.Sockets.Socket.Bind(EndPoint localEP)
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.SocketTransportOptions.CreateDefaultBoundListenSocket(EndPoint endpoint)
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.SocketConnectionListener.Bind()
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    --- End of inner exception stack trace ---
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.SocketConnectionListener.Bind()
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.SocketTransportFactory.BindAsync(EndPoint endpoint, CancellationToken cancellationToken)
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Infrastructure.TransportManager.BindAsync(EndPoint endPoint, ConnectionDelegate connectionDelegate, EndpointConfig endpointConfig, CancellationToken cancellationToken)
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.<>c__DisplayClass28_0`1.<<StartAsync>g__OnBind|0>d.MoveNext()
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z --- End of stack trace from previous location ---
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindEndpointAsync(ListenOptions endpoint, AddressBindContext context, CancellationToken cancellationToken)
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    --- End of inner exception stack trace ---
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindEndpointAsync(ListenOptions endpoint, AddressBindContext context, CancellationToken cancellationToken)
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Microsoft.AspNetCore.Server.Kestrel.Core.LocalhostListenOptions.BindAsync(AddressBindContext context, CancellationToken cancellationToken)
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.EndpointsStrategy.BindAsync(AddressBindContext context, CancellationToken cancellationToken)
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindAsync(ListenOptions[] listenOptions, AddressBindContext context, Func`2 useHttps, CancellationToken cancellationToken)
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.BindAsync(CancellationToken cancellationToken)
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.StartAsync[TContext](IHttpApplication`1 application, CancellationToken cancellationToken)
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Microsoft.AspNetCore.Hosting.GenericWebHostService.StartAsync(CancellationToken cancellationToken)
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Microsoft.Extensions.Hosting.Internal.Host.<StartAsync>b__15_1(IHostedService service, CancellationToken token)
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Microsoft.Extensions.Hosting.Internal.Host.ForeachService[T](IEnumerable`1 services, CancellationToken token, Boolean concurrent, Boolean abortOnFirstException, List`1 exceptions, Func`3 operation)
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Microsoft.Extensions.Hosting.Internal.Host.StartAsync(CancellationToken cancellationToken)
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.Run(IHost host)
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Aspire.Dashboard.DashboardWebApplication.Run() in /_/src/Aspire.Dashboard/DashboardWebApplication.cs:line 777
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Program.<Main>$(String[] args) in /_/src/Aspire.Dashboard/Program.cs:line 7
fail: Aspire.Hosting.Dashboard.Microsoft.Extensions.Hosting.Internal.Host[11]
      Hosting failed to start
      System.IO.IOException: Failed to bind to address https://127.0.0.1:17054: address already in use.
       ---> Microsoft.AspNetCore.Connections.AddressInUseException: Only one usage of each socket address (protocol/network address/port) is normally permitted.
       ---> System.Net.Sockets.SocketException (10048): Only one usage of each socket address (protocol/network address/port) is normally permitted.
         at System.Net.Sockets.Socket.UpdateStatusAfterSocketErrorAndThrowException(SocketError error, Boolean disconnectOnFailure, String callerName)
         at System.Net.Sockets.Socket.DoBind(EndPoint endPointSnapshot, SocketAddress socketAddress)
         at System.Net.Sockets.Socket.Bind(EndPoint localEP)
         at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.SocketTransportOptions.CreateDefaultBoundListenSocket(EndPoint endpoint)
         at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.SocketConnectionListener.Bind()
         --- End of inner exception stack trace ---
         at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.SocketConnectionListener.Bind()
         at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.SocketTransportFactory.BindAsync(EndPoint endpoint, CancellationToken cancellationToken)
         at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Infrastructure.TransportManager.BindAsync(EndPoint endPoint, ConnectionDelegate connectionDelegate, EndpointConfig endpointConfig, CancellationToken cancellationToken)
         at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.<>c__DisplayClass28_0`1.<<StartAsync>g__OnBind|0>d.MoveNext()
      --- End of stack trace from previous location ---
         at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindEndpointAsync(ListenOptions endpoint, AddressBindContext context, CancellationToken cancellationToken)
         --- End of inner exception stack trace ---
         at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindEndpointAsync(ListenOptions endpoint, AddressBindContext context, CancellationToken cancellationToken)
         at Microsoft.AspNetCore.Server.Kestrel.Core.LocalhostListenOptions.BindAsync(AddressBindContext context, CancellationToken cancellationToken)
         at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.EndpointsStrategy.BindAsync(AddressBindContext context, CancellationToken cancellationToken)
         at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindAsync(ListenOptions[] listenOptions, AddressBindContext context, Func`2 useHttps, CancellationToken cancellationToken)
         at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.BindAsync(CancellationToken cancellationToken)
         at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.StartAsync[TContext](IHttpApplication`1 application, CancellationToken cancellationToken)
         at Microsoft.AspNetCore.Hosting.GenericWebHostService.StartAsync(CancellationToken cancellationToken)
         at Microsoft.Extensions.Hosting.Internal.Host.<StartAsync>b__15_1(IHostedService service, CancellationToken token)
         at Microsoft.Extensions.Hosting.Internal.Host.ForeachService[T](IEnumerable`1 services, CancellationToken token, Boolean concurrent, Boolean abortOnFirstException, List`1 exceptions, Func`3 operation)

Expected Behavior

  1. I'd expect aspire to be able to recover from an orphaned dashboard process - ideally by not orphaning the process in the first place, but if not by killing it and re-starting.
  2. I also don't expect the stack trace above to be duplicated, with each line of the stack trace being it's own error.

Steps To Reproduce

End up with an orphaned Aspire.Dashboard.exe from one aspire run

Exceptions (if any)

No response

.NET Version info

.NET SDK:
 Version:           8.0.401
 Commit:            811edcc344
 Workload version:  8.0.400-manifests.57f7c351
 MSBuild version:   17.11.4+37eb419ad

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.22631
 OS Platform: Windows
 RID:         win-x64
 Base Path:   C:\Program Files\dotnet\sdk\8.0.401\

.NET workloads installed:
Configured to use loose manifests when installing new manifests.
 [aspire]
   Installation Source: VS 17.10.35201.131
   Manifest Version:    8.2.0/8.0.100
   Manifest Path:       C:\Program Files\dotnet\sdk-manifests\8.0.100\microsoft.net.sdk.aspire\8.2.0\WorkloadManifest.json
   Install Type:        FileBased

Host:
  Version:      8.0.8
  Architecture: x64
  Commit:       08338fcaa5

.NET SDKs installed:
  8.0.304 [C:\Program Files\dotnet\sdk]
  8.0.401 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 8.0.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 8.0.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 8.0.8 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Other architectures found:
  x86   [C:\Program Files (x86)\dotnet]
    registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x86\InstallLocation]

Environment variables:
  Not set

global.json file:
  Not found

Learn more:
  https://aka.ms/dotnet/info

Download .NET:
  https://aka.ms/dotnet/download

Anything else?

No response

davidfowl commented 2 months ago

Yes this is known. Are you using 8.2?

cc @karolz-ms

afscrome commented 2 months ago

Yup, I started seeing this after upgrading to 8.2.

davidfowl commented 2 months ago

This is an old bug, we just have much better logs now we can pinpoint the issue. I agree it would be ideal if we could not orphan the dashboard (we're not sure why this happens), and if we could kill the previous one if it was still running, that would require keeping state around somewhere...

karolz-ms commented 2 months ago

This is surprising; we made some extra changes to ensure nothing gets orphaned by accident in 8.2. @afscrome can you reproduce consistently?

afscrome commented 1 month ago

Sorry @karolz-ms haven't seen this much recently, but I've also been pulled away from aspire work for a bit. Feel free to close this and I can re-open if I encounter it again.

oising commented 1 month ago

Can confirm this is still happening in Aspire 8.2.1 (seen on amd64 & arm64) /cc @karolz-ms We cannot repro consistently. It just eventually happens after a sufficient amount of stop/start debugging sessions.

Moerup commented 1 month ago

Having the same problem, but it feels pretty damn consistent. I have to clean/rebuild between all successful runs before it kills the dashboard. Every second attempt to start AppHost up fails with a port in use error for the dashboard.

davidfowl commented 1 month ago

What version of aspire are you using ?

Moerup commented 1 month ago

Did not happen with 8.2.1, upgraded to 9 rc1 then it started happening, tried the nightly builds, same thing. Implementing IDisposable on the AppHost fixed it for me.

davidfowl commented 1 month ago

Are you using AddNpmApp?

Moerup commented 1 month ago

I'm using a few dapr sidecars and components, redis, sqlserver and azure storage emulator and 3 web apis + Blazor Web App.