dotnet / aspnetcore

ASP.NET Core is a cross-platform .NET framework for building modern cloud-based web applications on Windows, Mac, or Linux.
https://asp.net
MIT License
35.55k stars 10.05k forks source link

Static Web Assets cache busting mechanism #31922

Closed stefanloerwald closed 10 months ago

stefanloerwald commented 3 years ago

Is your feature request related to a problem? Please describe.

Browser cache files (partially depending on the server settings), which can lead to the situation that a scoped CSS file changed, but this is not picked up, because the file name didn't change. Replacing ProjectName.styles.css by ProjectName.styles.css?hash=123 doesn't produce the usual cache-busting mechanism, because the contents of that file (e.g. @import _content/library/library.bundle.scp.css) didn't change, only the file referenced by the import changed in content.

Describe the solution you'd like

Would it be possible to automatically calculate a file hash of the scoped CSS parts and import them with @import '_content/library/library.bundle.scp.css?hash=123';?

javiercn commented 3 years ago

@stefanloerwald thanks for contacting us.

We don't currently have a solution for this, however you can achieve this today with a middleware that checks for requests to files with the right extensions and adds the appropriate caching headers.

ghost commented 3 years ago

We've moved this issue to the Backlog milestone. This means that it is not going to be worked on for the coming release. We will reassess the backlog following the current release and consider this item at that time. To learn more about our issue management process and to have better expectation regarding different types of issues you can read our Triage Process.

mkArtakMSFT commented 3 years ago

@stefanloerwald can you please also share how you're hosting your site? Is it Blazor WebAssembly of Server project? Also, are you looking for this for dev-time or post publishing?

stefanloerwald commented 3 years ago

Certainly! I'm actually maintaining a few sites with different hosting models. The one that this is causing an inconvenience at the moment is a Blazor server project behind an nginx reverse proxy. The caching headers are set by nginx for the static assets, so I need to work client-side to make sure I fetch the new version.

The solution I proposed doesn't seem too complicated to implement, and I would even try to do it myself, if there is consensus that it's a good idea without negative impact for users. What does the blazor team think about it?

rogihee commented 3 years ago

Just FYI, I'm also running into this issue. This is for production in a Blazor WebAssembly hosted (multiple apps) solution, with scoped CSS coming from private feed NuGet Razor Class libraries.

XmlmXmlmX commented 2 years ago

This is a must have feature. I can't understand why this wasn't implemented by default from the start.

taylorchasewhite commented 2 years ago

Just FYI, I'm also running into this issue. This is for production in a Blazor WebAssembly hosted (multiple apps) solution, with scoped CSS coming from private feed NuGet Razor Class libraries.

Do you have any recommendations and/or examples of these middleware providers in use? Thanks for keeping this on the backlog, but looking to solve this as painlessly as possible in the meantime!

petterhoel commented 2 years ago

I am currently iterating fast on an application. Deploying daily, with feedback from our users. Forcing users to do hard refresh is a strong inconvenience for us at the moment.

We are on .NET 6 and using the server hosted WASM model.

Same as Taylor, pointers to the middleware approach would be highly appreciated and a built in option seems like a must in the long term.

NoahStolk commented 2 years ago

I'm also on .NET 6 using the server-hosted WASM model, and I'd be interested in the middleware approach as well. Currently need to do a hard refresh for every change in my CSS.

Rhywden commented 2 years ago

Count me in as well. Bonus: Edge on iOS does not even clear the cache even when supposedly forced to do so through settings so my Edge on iOS currently serves up an inoperable page (Safari at least properly clears the cache).

petterhoel commented 2 years ago

I am currently using this hackaround :

// This is in our our main layout file: 
<HeadContent>
    <link href="css/app.css?cacheBust=@_buster.GenerateCacheBustValue()" rel="stylesheet"/>
    <link href="AppName.Client.styles.css?cacheBust=@_buster.GenerateCacheBustValue()" rel="stylesheet"/>
</HeadContent>

@_buster.GenerateCacheBustValue() returns:

On local dev machine this causes flicker/layout jank on each navigation 😿. But at least our users have the latest stuff, don't need any hard refresh and flicker/layout jank for them is only once per version.

rogihee commented 2 years ago

@petterhoel I use a similar approach (I have hosted apps so server knows version) but the issue is if you have Razor Class Libraries the AppName.styles.css imports the RCL libs CSS without cache busting urls.

petterhoel commented 2 years ago

Yes there are many variations of this issue. Hopefully we will see some cache lifetime strategies for assets in the .net 7 time frame from the framework. Hashed file names, based on not the source code content (hash only changes if source changes), would be awesome.

dennisrahmen commented 2 years ago

@mkArtakMSFT Is this still planned for .NET 7?

javiercn commented 2 years ago

Unfortunately, this won't make the cut for .NET 7.0

philip-reed commented 1 year ago

Does this issue also impact blazor.webassembly.js?

I recently migrated a Blazor Wasm app from net6.0 to net7.0 and the app failed to load until a hard refresh cleared this from cache.

image

ghost commented 1 year ago

We've moved this issue to the Backlog milestone. This means that it is not going to be worked on for the coming release. We will reassess the backlog following the current release and consider this item at that time. To learn more about our issue management process and to have better expectation regarding different types of issues you can read our Triage Process.

higty commented 1 year ago

Is there any workaround to update webassembly cache?

gulbanana commented 1 year ago

Is there any workaround to update webassembly cache?

Sure, just do the cachebusting in javascript. In your index.html, remove the <link> and add something like this at the bottom of the <body>:

    <script>
        var link = document.createElement('link');
        link.type = 'text/css';
        link.rel = 'stylesheet';
        link.href = 'MyProject.styles.css?cache=' + new Date().getTime();
        document.head.appendChild(link);
    </script>
rogihee commented 1 year ago

@gulbanana that is not a full solution because the real issue is inside the *.styles.css file with import statements to other CSS files from dependent libraries, without a cache busting parameter.

davepermen commented 1 year ago

Definitely would be good to see a holistic approach to this issue, as it creeps up in all sort of places and would be easy to prevent well from the framework directly. it's hard to patchwork it as an enduser nicely.

(atm having clients that have the wrong css loaded on mobile, where a hard refresh is kinda impossible to do easily)

MichelJansson commented 1 year ago

Since this unfortunately got cut, yet again, I developed this simple workaround for scoped css from various ideas to solve this issue. It's not optimal, but not too far from it for my use case at least. It's so simple and unintrusive and I can't for the life of me understand why something similar isn't implemented by default.

  1. Add cache busting query param to the root css bundle. E Z.

    <link rel="stylesheet" href="{ASSEMBLY NAME}.styles.css" asp-append-version="true" />
  2. Add MSBuild task to the blazor client .csproj to re-write the bundles @import url's with cache busting query params.

    <!-- Task to add cache busting timestamps to scoped css @imports of library css bundles -->
    <Target Name="CachebustScopedCssImportsTask" AfterTargets="BundleScopedCssFiles"
         Condition="'@(_ScopedCss)' != ''">
     <PropertyGroup>
       <File>$(_ScopedCssOutputFullPath)</File>
       <Timestamp>$([System.DateTimeOffset]::Now.ToUnixTimeSeconds())</Timestamp>
     </PropertyGroup>
     <WriteLinesToFile
         File="$(File)"
         Lines="$([System.IO.File]::ReadAllText($(File)).Replace('.bundle.scp.css','.bundle.scp.css?v=$(Timestamp)'))"
         Overwrite="true"
         Encoding="UTF-8"/>
    </Target>

    This will add the unix timestamp of the build to each import as a query parameter to force the browsers to re-check the css bundles after each deployment. A hash would of course be better, but this is good enough for me as the normal etag cache check will still kick in and spare the browser from re-downloading the file if it was unchanged via a http 304.

jirisykora83 commented 12 months ago

Is there any workaround to update webassembly cache?

setup wasm as autostart false

<script asp-src-include="/_framework/blazor.webassembly.js" asp-append-version="true" autostart="false"></script>

and manually add integrity to query string to ensure that most (any modern?) of the browser load new resource if query string change.

This snippet is from asp hosted app @(ThisAssemblyInfo.GitCommitId) need to be replace for static hosted.

<script type="module">
    Blazor.start({
        loadBootResource: function (type, name, defaultUri, integrity) {
            let resourceVersion = integrity == null || integrity === '' ? '@(ThisAssemblyInfo.GitCommitId)' : integrity;
            if (type !== 'dotnetjs') {
                return (async function () {
                                const response = await fetch(`${defaultUri}?v=${resourceVersion}`);
                                if (!response.ok) {
                                    throw new Error(response.statusText);
                                }
                                return response;
                            }
                })();
            }

            return `${defaultUri}?v=${resourceVersion}`;
        },
        environment: "@this.Model.CurrentEnvironment"
    }).then(function () {
    });
</script>

Also, with this you actually can force for example cloduflare to cache .wasm / .dll files as there are by default not cache by CDN and always requested from origin which add latency for that use-case.

ghost commented 11 months ago

Thanks for contacting us.

We're moving this issue to the .NET 9 Planning milestone for future evaluation / consideration. We would like to keep this around to collect more feedback, which can help us with prioritizing this work. We will re-evaluate this issue, during our next planning meeting(s). If we later determine, that the issue has no community involvement, or it's very rare and low-impact issue, we will close it - so that the team can focus on more important and high impact issues. To learn more about what to expect next and how this issue will be handled you can read more about our triage process here.

mkArtakMSFT commented 10 months ago

We're going to address this as part of https://github.com/dotnet/aspnetcore/issues/52824

razblack commented 5 months ago

I've also run into this issue, but the problem is deeper than just simple CSS files... Javascript files are also not getting cache-busted and is becoming very troublesome to try and figure out a workaround.

I've upgraded to .NET 8 from Blazor Server App to Blazor Web App... full component based, which is nice and allows some magic to happen that helps cache busting.

in my use case, something as simple as site.css was really problematic in .NET6/7, and was easy to fix:

<link href="css/site.css?v=@(Guid.NewGuid().ToString())" rel="stylesheet" />

with .NET 8, moving all _Host.cstml to App.razor, including partial class App.razor.cs, I can now utilize a variable for this ie:

<link href="css/site.css?v=@SiteCSSVersion" rel="stylesheet" />

//and in App.razor.cs do:

string SiteCSSVersion = Guid.NewGuid().ToString();

However, I utilize some Javascript class libraries, and leverage on demand loading in OnAfterRenderAsync() of a component:

myService = await JSRuntime.InvokeAsync<IJSObjectReference>("import", $"./scripts/MyService.js?v={ServiceVersion}");

//where ServiceVersion as a cascaded parameter created in App.razor.cs, very similar to how site.css got versioned.

Unfortunately, this does not work in OnAfterRender, as the propagation of the cascaded value happens after firstRender.

Hooking it up in OnParameterSet can work to bust caching on that Javascript file, but then i run into other problems where that Javascript has imports.... ie:

import LibraryA from './LibraryA.js';
import LibraryB from './LibraryB.js';

I can't get the cache busted versions embedded into MyServer.js.. i have no clue how to do that. They all need to be busted.

So, I end up with something that looks like this being loaded to the browser scripts and css files:

./css
    /site.css?v=d2cbe0d7-ff97-46d0-b4c1-b4bd6d9a9900
./scripts
   /MyService?v=924929cc-dfba-41be-8748-de92dfab2e4b
   /LibraryA.js
   /LibraryA.js?v=924929cc-dfba-41be-8748-de92dfab2e4b
  /LibraryB.js
  /LibraryB.js?v=66155093-608a-429b-8ec0-fc5ab21efd7b

I cannot figure out a workaround, and I think this is a much more systematic problem. I can't ask hundreds of users to CTRL-F5 their browsers muliple times a week.

philip-reed commented 5 months ago

@razblack After running into this issue myself again this week, I've taken the following approach which is described here (though without examples):

https://learn.microsoft.com/en-us/aspnet/core/blazor/host-and-deploy/webassembly-caching/http-caching-issues?view=aspnetcore-8.0

The Cache-Control header is used to tell the browser not to cache anything from now on, but this doesn't help if you're updating an app that the browser may already have cached files for.

The Clear-Site-Data header solves this and can be used to tell browsers to clear any existing cached files (and cookies etc if you like), but you only want this to happen once during your application loading, otherwise, the browser will clear the cache for each request/response.

Technically you don't need the Cache-Control header if you're using the Clear-Site-Data header for each request to root, but this has burned me too many times, so I'm being extra cautious.

Using these headers, you also don't need to add versioning to css/js files linked in index.html, assuming they are served from the same domain as your app.

The code below is an extension method that adds middleware to add these headers for each request to the server. I'm using a server-hosted WASM app here, so if that isn't your hosting model, you may need to add the headers some other way.

 public static IApplicationBuilder UseDefaultResponseHeaders(this IApplicationBuilder app, IWebHostEnvironment env) =>
       app.Use(async (context, next) =>
       {
           if (context.Request.Path.ToString() == "/") // you may need a different check here
           {
               //Only tell the browser to clear the cache on the request to root paths, not again for each dependency 
               context.Response.Headers["Clear-Site-Data"] = "\"cache\"";    //yes the double-quotes are required, made that mistake too
           }

           context.Response.Headers["blazor-environment"] =  env.EnvironmentName;
           context.Response.Headers["Cache-control"] = "max-age=0, no-cache, no-store";

           await next.Invoke();
       });

Then in program.cs

       WebApplication app = builder.Build();

       app.UseDefaultResponseHeaders(env);
razblack commented 5 months ago

@razblack After running into this issue myself again this week, I've taken the following approach which is described here (though without examples):

How on earth did i miss this?!? lol

Thank you @philip-reed , i will be working this 1st thing Monday morning.

razblack commented 5 months ago

@philip-reed

wanted to add, still testing, but header is updated and hotreload is definitely grabbing files and appears to be busting the cache.

I simplified the middleware a little as i didn't need environments...

public class CacheBusterMiddleware
{
    private readonly RequestDelegate _next;

    public CacheBusterMiddleware(RequestDelegate next)
    {
        _next = next;
    }

    public async Task InvokeAsync(HttpContext context)
    {
        var path = context.Request.Path;

        if (path.HasValue && path.Value == "/")
        {
            context.Response.Headers["Clear-Site-Data"] = "\"cache\"";
        }
        await _next(context);
    }
}

public static class CacheBusterMiddlewareExtensions
{
    public static IApplicationBuilder UseCacheBusterMiddleware(this IApplicationBuilder builder)
    {
        return builder.UseMiddleware<CacheBusterMiddleware>();
    }
}

then added this before any redirection, routing, or auth:

app.UseCacheBusterMiddleware();

davepermen commented 5 months ago

I guess this could be combined somehow nicely with the actual deploy-version stored in the cookie or localstorage, and send the Clear-Site-Data only after deployment changed.

davepermen commented 5 months ago

this seems to do the trick to fully reload after a fresh deployment. reads build-hash from executable, based on https://stackoverflow.com/a/3634544 (it's not a timestamp nowadays, it's a hash)

stores application hash in a cookie, used in Program.cs as app.UseCompiledHashCacheBuster("app-hash-cookiename").


public static class CacheBusterExtension
{
    private static int? compiledHash;
    public static int CompiledHash => compiledHash ??= RetrieveLinkerHash();
    public static string CompiledHashString => CompiledHash.ToString("X");

    // http://www.codinghorror.com/blog/2005/04/determining-build-date-the-hard-way.html
    private static int RetrieveLinkerHash()
    {
        const int peHeaderOffset = 60;
        const int linkerCompileHashOffset = 8;
        var b = new byte[2048];
        FileStream s = null;
        try
        {
            s = new FileStream(System.Reflection.Assembly.GetExecutingAssembly().Location, FileMode.Open, FileAccess.Read);
            s.Read(b, 0, 2048);
        }
        finally
        {
            s?.Close();
        }
        return BitConverter.ToInt32(b, BitConverter.ToInt32(b, peHeaderOffset) + linkerCompileHashOffset);
    }

    public static IApplicationBuilder UseCompiledHashCacheBuster(this IApplicationBuilder app, string cookieName) => app.Use(async (context, next) =>
    {
        if (context.Request.Cookies.ContainsKey(cookieName) == true)
        {
            var hash = context.Request.Cookies[cookieName];
            if (hash != CompiledHashString)
            {
                context.Response.Headers["Clear-Site-Data"] = "\"cache\"";
            }
        }
        context.Response.Cookies.Append(cookieName, CompiledHashString, new()
        {
            Secure = true,
            Expires = DateTimeOffset.MaxValue

        });
        await next();
    });
}