Dotnet 9 Crash on Linux only

HugoVG commented 2 days ago

Describe the bug

YARP on Dotnet 9 crashes on Linux-x64

YARP attempts to use either IIS or HttpSys on linux machine in net9, but just works as intended on net8. same package version just different runtime, strange enough the application works just fine under Docker, but doesn't work under WSL and native Ubuntu Server(24). I originally caught it because the CICD tests were all failing after a dotnet 9 update, but they all passed on Windows (tested multiple workstation).

To Reproduce

Further technical details

.NET SDK: Version: 9.0.100 Commit: a2bc464e40 Workload version: 9.0.100-manifests.6bf02610 MSBuild version: 17.12.7+a2bc464e4

Runtime Environment: OS Name: ubuntu OS Version: 22.04 OS Platform: Linux RID: ubuntu.22.04-x64 Base Path: /usr/lib/dotnet/sdk/9.0.100/

.NET workloads installed: There are no installed workloads to display. Configured to use loose manifests when installing new manifests.

Host: Version: 9.0.0 Architecture: x64 Commit: a2bc464e40

.NET SDKs installed: 9.0.100 [/usr/lib/dotnet/sdk]

.NET runtimes installed: Microsoft.AspNetCore.App 9.0.0 [/usr/lib/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.NETCore.App 9.0.0 [/usr/lib/dotnet/shared/Microsoft.NETCore.App]

Include the version of the packages you are using Version 2.2.0 (worked under net8 no longer under net9)
The platform (Linux/macOS/Windows) Linux

Settings json

"ReverseProxy": {
    "Routes": {
        "all": {
            "ClusterId": "cluster1",
            "AuthorizationPolicy": "anonymous",
            "Match": {
                "Path": "{**catch-all}"
            },
            "Order": 999
        }
    },
    "Clusters": {
        "cluster1": {
            "Destinations": {
                "destination1": {
                    "Address": "https://<<FallbackServer>>"
                }
            }
        }
    }
},

MihaZupan commented 2 days ago

I can't repro this on WSL.

YARP attempts to use either IIS or HttpSys on linux machine in net9

Are you able to share error messages / stack traces / crash dumps?

HugoVG commented 1 day ago

Oh I'm stupid i thought i added the stack trace.

For the time being I've wrapped Yarp

if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
{
    app.MapReverseProxy();
}

Running on Kestrel
info: Program[0]
      Building services took 6755ms
info: Program[0]
      Current environment: Development
info: Program[0]
      Adding common Middleware took 19ms
info: Program[0]
      Mapping swagger took 195ms
info: Program[0]
      Setting up routing, Auth and controllers took 53ms
Unhandled exception. System.TypeLoadException: Could not load type 'Microsoft.AspNetCore.Server.HttpSys.IServerDelegationF
eature' from assembly 'Microsoft.AspNetCore.Server.HttpSys, Version=9.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'.
   at Yarp.ReverseProxy.Delegation.HttpSysDelegator..ctor(IServer server, ILogger`1 logger)
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)      
   at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)
   at System.Reflection.MethodBaseInvoker.InvokeWithFewArgs(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.CreateServiceAccessor(ServiceIdentifier serviceIdentifier) 
   at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(ServiceIdentifier serviceIdentifier, ServiceProviderEngineScope serviceProviderEngineScope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType)      
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
   at Yarp.ReverseProxy.Management.IReverseProxyBuilderExtensions.<>c.<AddHttpSysDelegation>b__8_1(IServiceProvider p)    
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitFactory(FactoryCallSite factoryCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitIEnumerable(IEnumerableCallSite enumerableCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.CreateServiceAccessor(ServiceIdentifier serviceIdentifier) 
   at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(ServiceIdentifier serviceIdentifier, ServiceProviderEngineScope serviceProviderEngineScope)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
   at Microsoft.AspNetCore.Builder.ReverseProxyIEndpointRouteBuilderExtensions.GetOrCreateDataSource(IEndpointRouteBuilder endpoints)
   at Microsoft.AspNetCore.Builder.ReverseProxyIEndpointRouteBuilderExtensions.MapReverseProxy(IEndpointRouteBuilder endpoints, Action`1 configureApp)
   at Microsoft.AspNetCore.Builder.ReverseProxyIEndpointRouteBuilderExtensions.MapReverseProxy(IEndpointRouteBuilder endpoints)
   at Program.<Main>$(String[] args) in /<pathToProgram>/Program.cs:line 189
   at Program.<Main>(String[] args)

MihaZupan commented 1 day ago

YARP will attempt to fetch the IServerDelegationFeature at startup, but it won't throw if one doesn't exist (exactly because you might be running on Linux / using Kestrel) https://github.com/microsoft/reverse-proxy/blob/ed43822e2fc0d6006f3ea4085ee48118bc173514/src/ReverseProxy/Delegation/HttpSysDelegator.cs#L36

System.TypeLoadException

This indicates that something went wrong when loading the Microsoft.AspNetCore.Server.HttpSys assembly (sadly not super informative as to the reason), and is outside of YARP's control.

Since I can't repro the same issue, it's possibly affected by something in your environment (how you installed dotnet, how you're packaging the app (self-contained, trimming, ...), any special monitoring/AV software, ...).

Can you please file an issue in https://github.com/dotnet/aspnetcore?

MihaZupan commented 1 day ago

Are you publishing the app as self-contained/with trimming/native AOT? Things like that could affect type loading.

HugoVG commented 15 hours ago

No they might be published self contained but normal dotnet run without AOT or trimming (sadly app is not compatible)

microsoft / reverse-proxy