HTTP2 Push Header - Feature Suggestion

[2aces]

Hey, @futtta . Celso Bessa, AO pt-br translator here.

As you know, HTTP2 push is a big deal in optimization now. We are getting great results in some of our projects with it and I think it would be great if AO included a mechanism to use those headers.

In some test setups we filtered AO final JS/CSS srcs and created our own headers using PHP/WP send headers ( https://codex.wordpress.org/Plugin_API/Action_Reference/send_headers ). this is not hard for us, but it might be not easy for an non technical user.

In other setups we used AO alongside http2 server push plugin by @daveros ), which is great, but it sends headers for all files in the original wp_enqueue queue but not for AO aggregated files (my guess is it uses a hook triggered before AO).

Looking this 2 cases, seems to me that having this on AO would help a lot of you user base.

What do you think? I won't be able to code anything for the next 4 weeks, but if you think it's a good feature, I can work on this in august.

[trajano]

Many issues... it may not be fixed though unless the spec has made it explicit that it should also be for non-scripted resources. So far only script injected resources will not be double downloaded.

https://bugs.chromium.org/p/chromium/issues/detail?id=655698 https://github.com/ampproject/amphtml/issues/7076

[futtta]

more in: https://bugs.chromium.org/p/chromium/issues/detail?id=655698 https://github.com/w3c/preload/issues/80

But these focus on a correct as= attrib. missing, which was not the case for you was it @trajano ?

[trajano]

I think it is more that it is dismissed already regardless I found that we shouldn't try to do guess what needs to be sent automatically and instead rely on the actual theme developer (which is why I stated use the resulting data. Stripping it off and sending it as part of the header would have some benefit HTTP2 will package the header more efficiently.

[futtta]

Well, for a site using AO (which is our context after all) I think HTTP2-pushing the autoptimized CSS & JS (and jquery, if excluded) could make sense actually :-)

[trajano]

You cannot push jquery unless you defer loading jquery. It may impact the page speed more if you defer such a critical component.

However the interim of doing the link parsing will help.

Now one thing that would help a lot is to somehow send the headers and configure the server automatically. But that goes on the realm of caching.

On Apr 4, 2017 at 9:19 AM, <frank goossens (mailto:notifications@github.com)> wrote:

Well, for a site using AO (which is our context after all) I think HTTP2-pushing the autoptimized CSS & JS (and jquery, if excluded) could make sense actually :-)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub (https://github.com/futtta/autoptimize/issues/49#issuecomment-291496978), or mute the thread (https://github.com/notifications/unsubscribe-auth/AAGwIyRjaLcVbAPx4zN3aub8aRL3KtZcks5rskNngaJpZM4JRNhx).

]]>

[futtta]

You cannot push jquery unless you defer loading jquery.

that part I didn't understand. is that the bug/ limitation you stumbled upon, or just as the spec intented?

However the interim of doing the link parsing will help.

what do you mean by that? harvesting link's in the header looking for preload and pushing those? doesn't really apply in AO's context, as AO aggregates all linked CSS replacing it with between 1 and 3 (generally, depends on different media-attribs in the original links) files.

configure the server automatically

must be getting tired, but I don't understand that part either, would you care to elaborate?

[trajano]

Best is to write a small HTML file and see what happens. Basically if you use chrome it will show if it downloads twice in the logs.

I am not sure how well things would work but if you do a deferred load the scripts. Like

Document.write("<script..."/> it will not yield the error.

On Apr 4, 2017 at 12:48 PM, <frank goossens (mailto:notifications@github.com)> wrote:

You cannot push jquery unless you defer loading jquery.

that part I didn't understand. is that the bug/ limitation you stumbled upon, or just as the spec intented?

However the interim of doing the link parsing will help.

what do you mean by that? harvesting link's in the header looking for preload and pushing those? doesn't really apply in AO's context, as AO aggregates all linked CSS replacing it with between 1 and 3 (generally, depends on different media-attribs in the original links) files.

configure the server automatically

must be getting tired, but I don't understand that part either, would you care to elaborate?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub (https://github.com/futtta/autoptimize/issues/49#issuecomment-291561573), or mute the thread (https://github.com/notifications/unsubscribe-auth/AAGwI6LEQPkShKo37ZOXOOmSQSY5ABMJks5rsnRqgaJpZM4JRNhx).

]]>

[trajano]

I just created a Gist, the warnings does appear in Chrome 57 http://rawgit.com/trajano/a61654502d4aa6191d742766f87dc5fe/raw/4aaa942f8457513d60fd1a26d953fd847b2aaea2/preload-double-download.html

[futtta]

Indeed, but that's for in-HTML link-preload, question is if this would also (still) happen when (only) doing HTTP/2 push (i.e. in HTTP response header).

[PatTheMav]

FYI (and just because I haven't seen it being mentioned) - the Cloudflare plugin supports HTTP/2 push (activated using a wp-config directive) and adds headers and <link rel="preload" as="script|style"> entries in the <head> - of course it doesn't do any good with autoptimize enabled, but at least gleaning at the source might be of interest.. 🙂

[2aces]

Just pinging here to 1 - apologize for not being active in the discussion/development because of health and other issues 2- fantastic collaboration in this post/feature, guys.

[futtta]

@2aces: get better soon!

@PatTheMav : wonder if combining HTTP/2 push (i.e. link preload in HTTP response headers) and link rel=preload in HTML <head> would not result in unwanted behavior? Have you been able to test this?

[PatTheMav]

@futtta - not yet, as I'm primarily using Safari, so I'd have to check this in my Windows VM on Edge.. 😉

But their docs make it seem like it's a "enable & forget" feature:

https://support.cloudflare.com/hc/en-us/articles/115002816808-How-do-I-enable-HTTP-2-Server-Push-in-WordPress

The source code itself mentions the following about adding the <link> elements:

Render "resource hints" in the section of the page. These encourage preload/prefetch behavior when HTTP/2 support is lacking.

Also it's just adding the headers using PHP's header function after adding itself as a filter for the script_loader_src and style_loader_src events. In addition it's mentioning an 8kb header size limit for Cloudflare (and a 4kb header size limit for fastcgi) so it limits the size of the header field to 3072 bytes (including CLRFs).

I'm not too versed with Wordpress' innards, so I don't know if there is a more elegant way to pull this off, but it seems workable.

Edit: Of course there is the elephant-in-the-room question about jQuery and many themes' reliance on jQuery being fully loaded in the head.

[futtta]

@PatTheMav I think we would indeed need to HTTP/2-push jquery.js, yes. we'll also have to (re-)consider deferring JS. if HTTP/2-pushed, the JS would not render-blocking (although the execution might be, cfr. "async"-flag which makes downloading not render-blocking but the actual execution is) so deferring (which delays JS execution until very late in the process) might not be optimal.

options options options ... ;-)

[trajano]

Be careful with deferring jquery. We may get flash of unstyled content. Which can be prevented by display none and unhiding when document is loaded. At the expense of slower initial load.

On Fri, Apr 28, 2017 at 7:32 AM frank goossens notifications@github.com wrote:

@PatTheMav https://github.com/PatTheMav I think we would indeed need to HTTP/2-push jquery.js, yes. we'll also have to (re-)consider deferring JS. is HTTP/2-pushed loading the JS is not render-blocking (although the execution might be, cfr. "async"-flag which makes downloading not render-blocking but the actual execution is).

options options options ... ;-)

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/futtta/autoptimize/issues/49#issuecomment-297975328, or mute the thread https://github.com/notifications/unsubscribe-auth/AAGwI9xATxPtoKDKpJAqr1-eq2CskkCQks5r0c5EgaJpZM4JRNhx .

[futtta]

i never defer jquery.js, in ao's default config it's even excluded from optimization. the point i was trying to make is that we might want to http/2-push jquery.js though :)

[PatTheMav]

@trajano - I'd say it's common practice to exclude jQuery from any deferred loading (and is more or less the default in AO). But @futtta is right that HTTP/2 push might make deferring itself (or placing in the footer) obsolete.

Personally I have no idea how the JS execution stack works in that case though, especially in conjunction with the event loop. There's a reason why some jS still "wants" to be loaded and parsed in the <head>, also some JS makes quite "optimistic" assumptions about when and how it's being loaded.

[zytzagoo]

Required reading: https://jakearchibald.com/2017/h2-push-tougher-than-i-thought/

[futtta]

regarding cloudflare; relevant code is here

regarding article @zytzagoo shared; interesting they make a distiinction between push and preload, as (based on what I read) webservers also use the preload http response header to determine what to push?

[futtta]

although I'm mainly working on stablizing AO 2.2 (hope some of you have it running, need to be more or less confident it's OK before releasing), this code snippet (a proof of concept, nothing more) pushes the AO'ed files and jquery.js;

add_filter('autoptimize_filter_cache_getname','pushAOFiles');
function pushAOFiles($in) {
  $pushType = substr($in,strrpos($in,".")+1) === "js" ? "script" : "style"; 
  header('link: <'.$in.'>; rel=preload; type='.$pushType,false);
  return $in;
}

add_filter('autoptimize_filter_js_exclude','pushJQuery');
function pushJQuery($in) {
    if (strpos($in,"js/jquery/jquery.js")!==false) {
        $jQurl=includes_url("js/jquery/jquery.js");
        header('link: <'.$jQurl.'>; rel=preload; type=js',false);
    }
}

I think that if we're HTTP/2 pushing, we would also want to remove the defer flag from the autoptimized JS, OR go for "look only in head"+"force in head" for JS.

[zytzagoo]

My (simple) takeaway is to keep things on 1.1 still for now, it's pretty messy... and things can actually end up being slower.

But, really, the article is a must read. Especially about the preload/push distinction.

[grzegorz-janoszka]

I went through all this topic a few months ago and now. I have been using the code based on trajano's comment on 21 Aug 2016. It has been spitting quite a big header. I have removed the images and focused only on css and js and then I had (among others) such headers sent: <//pagead2.googlesyndication.com/pagead/js/adsbygoogle.js>; rel=preload; as=script,<//pagead2.googlesyndication.com/pagead/js/adsbygoogle.js>; rel=preload; as=script,<//pagead2.googlesyndication.com/pagead/js/adsbygoogle.js> ... and so on.

The new code from futta's comment on on 31 May 2017 displays only AO files.

Does the push specification allow for 3rd party files in the headers? Does it make sense at all?

[grzegorz-janoszka]

I took a look at what other popular sites are doing with server push and I noticed in the headers sent by akamai.com:

Link: https://www.googletagmanager.com;rel="preconnect",https://www.google-analytics.com;rel="preconnect",https://www.googleadservices.com;rel="preconnect"

So maybe it is not so bad idea? Is there any way to get also other js files by some smart filters in AO 2.2?

[futtta]

well, preconnect is not HTTP/2 push, it's "just" telling the browser to already create a HTTP-connection to those domains :-)

regarding how many files to HTTP/2 push; only time (and testing) will tell, but from what I've read, heard & discussed one would not want to push all files, but limit the push to those files that are needed for the immediate rendering and leave the rest to normal loading?

[trajano]

From what I can tell you should only preload items that are going to be loaded via script using the HTTP Header to get the proper optimization. If it is going to be loaded from the main HTML directly you would actually be wasting more bandwidth. This is with Chrome specifically.

One good use but may be too complex to implement is “placeholder” images. For example we can have low fidelity (as in a single pixel image) as a place holder scaled up to the size and have a script that would dynamically replace them with the proper images. In that case the images should be preloaded using HTTP push via the header.

Sent from Mail for Windows 10

From: frank goossens Sent: July 28, 2017 6:46 AM To: futtta/autoptimize Cc: Archimedes Trajano; Mention Subject: Re: [futtta/autoptimize] HTTP2 Push Header - Feature Suggestion (#49)

well, preconnect is not HTTP/2 push, it's "just" telling the browser to already create a HTTP-connection to those domains :-) regarding how many files to HTTP/2 push; only time (and testing) will tell, but from what I've read, heard & discussed one would not want to push all files, but limit the push to those files that are needed for the immediate rendering and leave the rest to normal loading? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[kevin25]

Do you have idea about this warning? Looks like the push server doesn't work out.

The resource /wp-content/themes/flatsome/assets/css/fl-icons.css?ver=3.3 was preloaded using link preload but not used within a few seconds from the window's load event. Please make sure it wasn't preloaded for nothing. 08:59:09.128 (index):1 The resource /wp-content/themes/flatsome/assets/css/flatsome.css?ver=3.3.8 was preloaded using link preload but not used within a few seconds from the window's load event. Please make sure it wasn't preloaded for nothing. 08:59:09.128 (index):1 The resource /wp-content/themes/flatsome-child/style.css?ver=3.3.8 was preloaded using link preload but not used within a few seconds from the window's load event. Please make sure it wasn't preloaded for nothing. 08:59:09.128 (index):1 The resource /wp-content/themes/flatsome/assets/css/flatsome-shop.css?ver=3.3.8 was preloaded using link preload but not used within a few seconds from the window's load event. Please make sure it wasn't preloaded for nothing.

[futtta]

@kevin25 see some of @trajano's replies above; looks like chrome will (re-)download HTTP2-pushed resources if those resources aren't added by JS, so preloading normally linked CSS will probably see Chrome consider those preloaded files as useless.

[kevin25]

@futtta So any solution for this?

[futtta]

well, you could try to have your CSS loaded by JS?

On Sat, Sep 2, 2017 at 6:06 PM, Kevin notifications@github.com wrote:

@futtta https://github.com/futtta So any solution for this?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/futtta/autoptimize/issues/49#issuecomment-326753273, or mute the thread https://github.com/notifications/unsubscribe-auth/AALEMcGLbpnGh5Y0O7ALWoeTUnPs4tHfks5seXz6gaJpZM4JRNhx .

[trajano]

You just need to know what the scripts would load (which is not really an easy thing to do in a general automated fashion)

Some candidates for this would be local fonts which can passed using preload and HTTP Push. Below-the-fold CSS can also be preloaded while the above the fold CSS is still part of the HTML that gets originally sent.

Lastly another candidate for preload would be the Ad network CSSes which are generally loaded via script.

In order for this to work well you need to have them part of the header rather than the body because if it is in the body then it will have to read a few bytes in and parse before it can do anything. You still need it in the body to actually be used by your page though.

The hack I did for this on an earlier version was to create a ".headers" file that gets sent if it is found and have a rule in the Apache configuration.

[kevin25]

@futtta Do you have any sample?

[futtta]

apart from the autoptimize-specific code I have no samples, no.

On Mon, Sep 4, 2017 at 2:26 PM, Kevin notifications@github.com wrote:

@futtta https://github.com/futtta Do you have any sample?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/futtta/autoptimize/issues/49#issuecomment-326950529, or mute the thread https://github.com/notifications/unsubscribe-auth/AALEMVKmAPj4vNpOYHJIsNMRt_HTZOn-ks5se-xugaJpZM4JRNhx .

[trajano]

@kevin25 http://www.javascriptkit.com/javatutors/loadjavascriptcss.shtml but I don't recommend it for anything above the fold. (i.e. anything shown on the first page before you scroll)

[grzegorz-janoszka]

I think the solutions in this post stopped working. I have:

add_filter('autoptimize_filter_cache_getname','pushAOFiles'); function pushAOFiles($in) { $pushType = substr($in,strrpos($in,".")+1) === "js" ? "script" : "style"; header('Link: <'.$in.'>; rel=preload; type='.$pushType,false); return $in; } add_filter('autoptimize_filter_js_exclude','pushJQuery'); function pushJQuery($in) { if (strpos($in,"js/jquery/jquery.js")!==false) { $jQurl=includes_url("js/jquery/jquery.js"); header('Link: <'.$jQurl.'>; rel=preload; type=js',false); } }

and it seems it doesn't work anymore. Any help with that highly appreciated.

[tdtgit]

The below function works with latest AO even.

function http2_server_push($content) {
    $header = "Link: ";
      if (preg_match('#="([^"]+/js/autoptimize_[0-9a-f]+\.js)"#', $content, $matches)) {
        $header .= sprintf(
            '<%s>; rel=preload; as=%s,',
               $matches[1], 'script'
          );
      }

      if (preg_match('#="([^"]+/css/autoptimize_[0-9a-f]+\.css)"#', $content, $matches)) {
        $header .=
          sprintf(
            '<%s>; rel=preload; as=%s',
               $matches[1], 'style'
          );
      }
      header($header);
      return $content;
}
add_filter('autoptimize_html_after_minify', 'http2_server_push');

[grzegorz-janoszka]

Indeed, but it is sooo heavy! I am looking for a more light solution.

[futtta]

what version are you testing against @grzegorz-janoszka ; 2.3.4 or 2.4-beta ?

[grzegorz-janoszka]

Ah, indeed that's handy information. I am using 2.3.4 still.

[futtta]

just tested on my local dev machine, works for me @grzegorz-janoszka ?

[tdtgit]

Hi @futtta, just wonder your preload script using type instead of as.

Timeline of using as, scripts and styles preloaded at top priority:

Timeline of using type, CSS files are not loaded even:

And maybe

header('link: <'.$jQurl.'>; rel=preload; type=js',false);

should be

header('link: <'.$jQurl.'>; rel=preload; as=script',false); too :)

[futtta]

absolutely @tdtgit :)

[grzegorz-janoszka]

@futta, thank you for testing. I noticed in the past that it sometimes worked, sometimes didn't. I checked all the headers called with Link: and they all have false as the second argument. I was thinking that maybe wordpress already output some data and thus the header call is void. I have another header call that always worked and it is at the 'template_redirect' level. What main level of actions/filters is the code I pasted called?

[tdtgit]

@grzegorz-janoszka Try my edited code here. @futta's code is running fine when adding Link preload to response header, but not correctly so browser isn't preloading resources at top priority.

[futtta]

it hooks into AO's autoptimize_filter_cache_getname (in autoptimizeCache.php) which is added in autoptimizeCache.php's getname() function which is called after CSS & JS minification as part of their cache() function.

[grzegorz-janoszka]

@tdtgit I updated my code with your suggestion, thank you. But my problem is that the headers are not sent. @futtta but on which level on the top-level action/filters is all that code happening?

[futtta]

Well, AO uses the output buffer, hooking into template_redirect by default and ending when wordpress flushes the OB.

On Thu, Apr 19, 2018 at 9:54 AM, grzegorz-janoszka <notifications@github.com

wrote:

@tdtgit https://github.com/tdtgit I updated my code with your suggestion, thank you. But my problem is that the headers are not sent. @futtta https://github.com/futtta but on which level on the top-level action/filters is all that code happening?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/futtta/autoptimize/issues/49#issuecomment-382644681, or mute the thread https://github.com/notifications/unsubscribe-auth/AALEMQKDptTiCnEDntfkE5B5zwZZhHdKks5tqEK4gaJpZM4JRNhx .

[tdtgit]

@grzegorz-janoszka Can you send me your site? @futtta HTTP/2 Push is a cool feature and it's needed when HTTPS and HTTP/2 is became more popular. When will we see it as an option in AO plugin?

[futtta]

so what do you reckon we should push Anh? indeed the AO'ed CSS & JS + jquery (if excluded)?

On Thu, Apr 19, 2018 at 10:00 AM, Anh Tuấn notifications@github.com wrote:

@grzegorz-janoszka https://github.com/grzegorz-janoszka Can you send me your site? @futtta https://github.com/futtta HTTP/2 Push is a cool feature and it's needed when HTTP/2 is became more popular. When will we see it as an option in AO plugin?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/futtta/autoptimize/issues/49#issuecomment-382646190, or mute the thread https://github.com/notifications/unsubscribe-auth/AALEMb370E1ii0fmIqskRS7kqfLOnkoTks5tqEQLgaJpZM4JRNhx .

[grzegorz-janoszka]

I just disabled all other plugins and stripped my own plugin to just those two AO filters and still nothing - no link headers sent by autoptimize_filter_cache_getname :( Really have no idea where to look for it. Yesterday I updated nginx on my site to support http2_push_preload and I would like to test it... and I can't :(

[futtta]

page caching issue?

On Thu, Apr 19, 2018 at 10:17 AM, grzegorz-janoszka < notifications@github.com> wrote:

I just disabled all other plugins and stripped my own plugin to just those two AO filters and still nothing - no link headers sent by autoptimize_filter_cache_getname :( Really have no idea where to look for it. Yesterday I updated nginx on my site to support http2_push_preload and I would like to test it... and I can't :(

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/futtta/autoptimize/issues/49#issuecomment-382650786, or mute the thread https://github.com/notifications/unsubscribe-auth/AALEMTlClQDfjVWaCkskSyEpsNdnLLnnks5tqEgqgaJpZM4JRNhx .