search

OrchardCMS / OrchardCore

Orchard Core is an open-source modular and multi-tenant application framework built with ASP.NET Core, and a content management system (CMS) built on top of that framework.
https://orchardcore.net
BSD 3-Clause "New" or "Revised" License
7.44k stars 2.4k forks source link

Investigate many static file middleware #1730

Open sebastienros opened 6 years ago

sebastienros commented 6 years ago

Here is a trace of all middleware called. Each module with a wwwroot folder create a new middleware, maybe it would be better to use a custom FileProvider instead.

Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType)
Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService<T>(IServiceProvider provider)
Microsoft.AspNetCore.Mvc.Internal.AttributeRoute.GetTreeRouter()
Microsoft.AspNetCore.Mvc.Internal.AttributeRoute.RouteAsync(RouteContext context)
Microsoft.AspNetCore.Routing.RouteCollection.RouteAsync(RouteContext context)
Microsoft.AspNetCore.Builder.RouterMiddleware.Invoke(HttpContext httpContext)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
SixLabors.ImageSharp.Web.Middleware.ImageSharpMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.Diagnostics.StatusCodePagesMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.Server.IISIntegration.IISMiddleware.Invoke(HttpContext httpContext)
OrchardCore.Modules.ModularTenantRouterMiddleware.Invoke(HttpContext httpContext) in ModularTenantRouterMiddleware.cs
OrchardCore.Modules.ModularTenantContainerMiddleware.Invoke(HttpContext httpContext) in ModularTenantContainerMiddleware.cs
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)
jtkech commented 6 years ago

Yes, we already do the following for the ContentRootFileProvider.

        env.ContentRootFileProvider = new CompositeFileProvider(
            new ModuleEmbeddedFileProvider(env),
            env.ContentRootFileProvider);

I will see if we can do something similar for the WebRootFileProvider.

marlon-tucker commented 6 years ago

I would also like to add that this bug is adding a fair amount of response time to our application.

Our use case is slightly edge case in that we are requesting files via a controller action, and the URL looks like a static file but is actually handled via a controller action. It would be great if Orchard modules could provide some way to exclude / include paths that are to be handled by the static file middleware. As in our case, every static file is in a sub folder inside the modules wwwroot folder.

Here is an example output from one request to the controller action in question:

info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1]
      Request starting HTTP/1.1 GET http://custa.localhost:36004/360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.Session.SessionMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: OrchardCore.Modules.PoweredByMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: OrchardCore.Modules.ModularTenantContainerMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: OrchardCore.Modules.ModularTenantRouterMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.Authentication.AuthenticationMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
info: Microsoft.AspNetCore.Authentication.Cookies.CookieAuthenticationHandler[8]
      AuthenticationScheme: Cookies was successfully authenticated.
MiddlewareStarting: Microsoft.AspNetCore.Diagnostics.StatusCodePagesMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.Builder.UseExtensions+<>c__DisplayClass0_1; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: SixLabors.ImageSharp.Web.Middleware.ImageSharpMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
MiddlewareStarting: Microsoft.AspNetCore.Builder.RouterMiddleware; /360.FirstTouchForm/Forms/GetFormFile/1stTouch.Forms.Rendering.js
info: Microsoft.AspNetCore.Authorization.DefaultAuthorizationService[1]
      Authorization was successful for user: e.
info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[1]
      Executing action method AareonUK360.WebApp.Modules.FirstTouchForm.Controllers.FormsController.GetFormFile (360.FirstTouchForm) with arguments (1stTouch.Forms.Rendering.js) - ModelState is Valid
info: Microsoft.AspNetCore.Authentication.Cookies.CookieAuthenticationHandler[8]
      AuthenticationScheme: Cookies was successfully authenticated.
info: Microsoft.AspNetCore.Mvc.StatusCodeResult[1]
      Executing HttpStatusCodeResult, setting HTTP status code 304
info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[2]
      Executed action AareonUK360.WebApp.Modules.FirstTouchForm.Controllers.FormsController.GetFormFile (360.FirstTouchForm) in 189.969ms
MiddlewareFinished: Microsoft.AspNetCore.Builder.RouterMiddleware; 304; Took 224.2341ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 237.3075ms
MiddlewareFinished: SixLabors.ImageSharp.Web.Middleware.ImageSharpMiddleware; 304; Took 248.4448ms
MiddlewareFinished: Microsoft.AspNetCore.Builder.UseExtensions+<>c__DisplayClass0_1; 304; Took 261.8116ms
MiddlewareFinished: Microsoft.AspNetCore.Diagnostics.StatusCodePagesMiddleware; 304; Took 269.8149ms
MiddlewareFinished: Microsoft.AspNetCore.Authentication.AuthenticationMiddleware; 304; Took 311.9082ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 333.4891ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 340.6988ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 367.7565ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 376.3359ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 397.1275ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 402.8105ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 415.7715ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 432.6143ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 446.8391ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 467.5753ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 474.7161ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 485.0956ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 503.0723ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 527.6253ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 531.8074ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 546.2489ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 553.1365ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 563.8782ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 568.8311ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 588.7456ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 599.0245ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 608.3154ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 619.8117ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 622.4265ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 637.5797ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 648.7286ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 658.0173ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 669.5854ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 680.993ms
MiddlewareFinished: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware; 304; Took 698.0383ms
MiddlewareFinished: OrchardCore.Modules.ModularTenantRouterMiddleware; 304; Took 703.7305ms
MiddlewareFinished: OrchardCore.Modules.ModularTenantContainerMiddleware; 304; Took 733.0103ms
MiddlewareFinished: OrchardCore.Modules.PoweredByMiddleware; 304; Took 743.9602ms
MiddlewareFinished: Microsoft.AspNetCore.Session.SessionMiddleware; 304; Took 749.3037ms
MiddlewareFinished: Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware; 304; Took 751.6945ms
info: Microsoft.AspNetCore.Hosting.Internal.WebHost[2]
      Request finished in 769.7883ms 304

The actual action completed in under 200ms, but due to all the middle ware that is in the pipeline, this caused a 770ms response time.

sebastienros commented 6 years ago

Instead of using a controller/action, is there a way for you to use a Map call to use a middleware or a lambda directly? Performance testing has shown that using controller/actions might have a significant impact on perf. That would also fix the issue you are decribing.

How do you get these stats for each middleware?

marlon-tucker commented 6 years ago

I used the middleware analysis tool and a diagnostic source listener. The middleware finished event can provide a duration.

I'll look into using a custom middleware to serve the files tomorrow. Thanks for the suggestion.

Marlon

On Thu, 7 Jun 2018, 18:15 Sébastien Ros, notifications@github.com wrote:

Instead of using a controller/action, is there a way for you to use a Map call to use a middleware or a lambda directly? Performance testing has shown that using controller/actions might have a significant impact on perf. That would also fix the issue you are decribing.

How do you get these stats for each middleware?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/OrchardCMS/OrchardCore/issues/1730#issuecomment-395497779, or mute the thread https://github.com/notifications/unsubscribe-auth/ABHOQ4UHSNlvoWacuUW7DqYunDpogXm0ks5t6V_AgaJpZM4Tj4RV .

sebastienros commented 6 years ago

@jtkech https://andrewlock.net/under-the-hood-of-the-middleware-analysis-package/

In case it give you some ideas, IStartupFilter

sebastienros commented 6 years ago

I suspect the tracing infrastructure (maybe middleware analysis tool also) is adding some cost for each middleware. You can test it by adding a no-op middleware, which shouldn't take multiple milliseconds.

jtkech commented 6 years ago

Okay, i will take a look. Just to say that in the netocoreapp21 branch we group all module static file providers in only one, included in the tenants branch.

Yes, here i think we also measure the analysis middleware called between each other middlewares.

sebastienros commented 6 years ago

so let's not do anything until netcoreapp21 branch is merged

gvkries commented 3 months ago

This doesn't apply anymore, does it?

github-actions[bot] commented 20 hours ago

We triaged this issue and set the milestone according to the priority we think is appropriate (see the docs on how we triage and prioritize issues).

This indicates when the core team may start working on it. However, if you'd like to contribute, we'd warmly welcome you to do that anytime. See our guide on contributions here.