search

OrchardCMS / OrchardCore

Orchard Core is an open-source modular and multi-tenant application framework built with ASP.NET Core, and a content management system (CMS) built on top of that framework.
https://orchardcore.net
BSD 3-Clause "New" or "Revised" License
7.41k stars 2.39k forks source link

Investigate many static file middleware #1730

Open sebastienros opened 6 years ago

sebastienros commented 6 years ago

Here is a trace of all middleware called. Each module with a wwwroot folder create a new middleware, maybe it would be better to use a custom FileProvider instead.

Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType)
Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService<T>(IServiceProvider provider)
Microsoft.AspNetCore.Mvc.Internal.AttributeRoute.GetTreeRouter()
Microsoft.AspNetCore.Mvc.Internal.AttributeRoute.RouteAsync(RouteContext context)
Microsoft.AspNetCore.Routing.RouteCollection.RouteAsync(RouteContext context)
Microsoft.AspNetCore.Builder.RouterMiddleware.Invoke(HttpContext httpContext)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
SixLabors.ImageSharp.Web.Middleware.ImageSharpMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.Diagnostics.StatusCodePagesMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.Server.IISIntegration.IISMiddleware.Invoke(HttpContext httpContext)
OrchardCore.Modules.ModularTenantRouterMiddleware.Invoke(HttpContext httpContext) in ModularTenantRouterMiddleware.cs
OrchardCore.Modules.ModularTenantContainerMiddleware.Invoke(HttpContext httpContext) in ModularTenantContainerMiddleware.cs
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)
jtkech commented 6 years ago

Yes, we already do the following for the ContentRootFileProvider.

        env.ContentRootFileProvider = new CompositeFileProvider(
            new ModuleEmbeddedFileProvider(env),
            env.ContentRootFileProvider);

I will see if we can do something similar for the WebRootFileProvider.

marlon-tucker commented 6 years ago

I would also like to add that this bug is adding a fair amount of response time to our application.

Our use case is slightly edge case in that we are requesting files via a controller action, and the URL looks like a static file but is actually handled via a controller action. It would be great if Orchard modules could provide some way to exclude / include paths that are to be handled by the static file middleware. As in our case, every static file is in a sub folder inside the modules wwwroot folder.

Here is an example output from one request to the controller action in question:

info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1]
      Request starting HTTP/1.1 GET http://custa.localhost:36004/360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.Session.SessionMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: OrchardCore.Modules.PoweredByMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: OrchardCore.Modules.ModularTenantContainerMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: OrchardCore.Modules.ModularTenantRouterMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.Authentication.AuthenticationMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
info: Microsoft.AspNetCore.Authentication.Cookies.CookieAuthenticationHandler[8]
      AuthenticationScheme: Cookies was successfully authenticated.
MiddlewareStarting: Microsoft.AspNetCore.Diagnostics.StatusCodePagesMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.Builder.UseExtensions+<>c__DisplayClass0_1; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: SixLabors.ImageSharp.Web.Middleware.ImageSharpMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.Builder.RouterMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
info: Microsoft.AspNetCore.Authorization.DefaultAuthorizationService[1]
      Authorization was successful for user: e.
info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[1]
      Executing action method AareonUK360.WebApp.Modules.FirstTouchForm.Controllers.FormsController.GetFormFile (360.FirstTouchForm) with arguments (1stTouch.Forms.Rendering.js) - ModelState is Valid
info: Microsoft.AspNetCore.Authentication.Cookies.CookieAuthenticationHandler[8]
      AuthenticationScheme: Cookies was successfully authenticated.
info: Microsoft.AspNetCore.Mvc.StatusCodeResult[1]
      Executing HttpStatusCodeResult, setting HTTP status code 304
info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[2]
      Executed action AareonUK360.WebApp.Modules.FirstTouchForm.Controllers.FormsController.GetFormFile (360.FirstTouchForm) in 189.969ms
MiddlewareFinished: Microsoft.AspNetCore.Builder.RouterMiddleware; 304; Took 224.2341ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 237.3075ms
MiddlewareFinished: SixLabors.ImageSharp.Web.Middleware.ImageSharpMiddleware; 304; Took 248.4448ms
MiddlewareFinished: Microsoft.AspNetCore.Builder.UseExtensions+<>c__DisplayClass0_1; 304; Took 261.8116ms
MiddlewareFinished: Microsoft.AspNetCore.Diagnostics.StatusCodePagesMiddleware; 304; Took 269.8149ms
MiddlewareFinished: Microsoft.AspNetCore.Authentication.AuthenticationMiddleware; 304; Took 311.9082ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 333.4891ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 340.6988ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 367.7565ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 376.3359ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 397.1275ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 402.8105ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 415.7715ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 432.6143ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 446.8391ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 467.5753ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 474.7161ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 485.0956ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 503.0723ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 527.6253ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 531.8074ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 546.2489ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 553.1365ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 563.8782ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 568.8311ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 588.7456ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 599.0245ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 608.3154ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 619.8117ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 622.4265ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 637.5797ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 648.7286ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 658.0173ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 669.5854ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 680.993ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 698.0383ms
MiddlewareFinished: OrchardCore.Modules.ModularTenantRouterMiddleware; 304; Took 703.7305ms
MiddlewareFinished: OrchardCore.Modules.ModularTenantContainerMiddleware; 304; Took 733.0103ms
MiddlewareFinished: OrchardCore.Modules.PoweredByMiddleware; 304; Took 743.9602ms
MiddlewareFinished: Microsoft.AspNetCore.Session.SessionMiddleware; 304; Took 749.3037ms
MiddlewareFinished: Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware; 304; Took 751.6945ms
info: Microsoft.AspNetCore.Hosting.Internal.WebHost[2]
      Request finished in 769.7883ms 304

The actual action completed in under 200ms, but due to all the middle ware that is in the pipeline, this caused a 770ms response time.

sebastienros commented 6 years ago

Instead of using a controller/action, is there a way for you to use a Map call to use a middleware or a lambda directly? Performance testing has shown that using controller/actions might have a significant impact on perf. That would also fix the issue you are decribing.

How do you get these stats for each middleware?

marlon-tucker commented 6 years ago

I used the middleware analysis tool and a diagnostic source listener. The middleware finished event can provide a duration.

I'll look into using a custom middleware to serve the files tomorrow. Thanks for the suggestion.

Marlon

On Thu, 7 Jun 2018, 18:15 Sébastien Ros, notifications@github.com wrote:

Instead of using a controller/action, is there a way for you to use a Map call to use a middleware or a lambda directly? Performance testing has shown that using controller/actions might have a significant impact on perf. That would also fix the issue you are decribing.

How do you get these stats for each middleware?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/OrchardCMS/OrchardCore/issues/1730#issuecomment-395497779, or mute the thread https://github.com/notifications/unsubscribe-auth/ABHOQ4UHSNlvoWacuUW7DqYunDpogXm0ks5t6V_AgaJpZM4Tj4RV .

sebastienros commented 6 years ago

@jtkech https://andrewlock.net/under-the-hood-of-the-middleware-analysis-package/

In case it give you some ideas, IStartupFilter

sebastienros commented 6 years ago

I suspect the tracing infrastructure (maybe middleware analysis tool also) is adding some cost for each middleware. You can test it by adding a no-op middleware, which shouldn't take multiple milliseconds.

jtkech commented 6 years ago

Okay, i will take a look. Just to say that in the netocoreapp21 branch we group all module static file providers in only one, included in the tenants branch.

Yes, here i think we also measure the analysis middleware called between each other middlewares.

sebastienros commented 6 years ago

so let's not do anything until netcoreapp21 branch is merged

gvkries commented 2 months ago

This doesn't apply anymore, does it?