search

andydavies / http2-prioritization-issues

Tracks issues / notes for HTTP/2 prioritization across browsers, CDNs and servers
MIT License
265 stars 7 forks source link

Feedback: request order and stream id (test vs. Apache httpd) #9

Closed icing closed 5 years ago

icing commented 5 years ago

Summary

Thanks for this wonderful test. I just had to check my Apache implementation with it.

The nice waterfall pictures rendered have the shortcoming that stream id's are not reflected in the order presented in the picture. For some tests, this works out, for others the waterfall is more a sawtooth - although basically the same things happened.

That might be a shortcoming of webpagetest itself. Can you recommend a viewer that does this better? Maybe using the .har file one can export? Would make analyzing this much easier.

Server / CDN etc.

From: Nottingham, East Midlands, UK - Chrome - 3G 19.3.2019, 10:12:53

Browser

Chrome

Link to WebPageTest result

https://www.webpagetest.org/result/190319_XT_ef9491d8583f3eaef8070c33f73ca6e3/9/details/#waterfall_view_step1

Link to upstream bug report

pmeenan commented 5 years ago

Thanks, I'll take a look. It has been bugging me for a while but I haven't done anything about it yet. My guess is that requests issues in the same millisecond get sorted randomly since they are sorted by start time and I can add stream ID as a second-order sort.

FWIW, for this specific test it doesn't matter. The last 2 requests are the important ones and are guaranteed to be sorted correctly since they are intentionally issued after a delay (and are serialized).

pmeenan commented 5 years ago

The actual ordering should be fixed now. I'm keeping the full-resolution timestamps for sorting purposes.

That said, I was wrong about it being the last 2 requests that are always the interesting ones. That assumes a max concurrent stream count that is sufficiently large. In this case it looks like it is limited to 10 concurrent streams so Chrome re-prioritizes the requests before asking for them.

tunetheweb commented 5 years ago

Interesting...It looks like Chrome is limiting the number of requests it sends when in 3G mode. It doesn't do this when on a fast connection.

And investigating a little deeper, it is not doing this by throttling SETTINGS_MAX_CONCURRENT_STREAMS as that is still set to 1000 in both cases from the Browser and 100 from the server when using Apache.

It didn't used to do this when I tested Apache previously, back in November: https://twitter.com/tunetheweb/status/1065699332064780289 https://www.webpagetest.org/result/181122_5J_14327e2f4c3b24eee9bfa910a8dbb6ab/ So this is something new.

And it doesn't do it in Chrome Canary either: https://www.webpagetest.org/result/190319_VT_7aa5dfe3dc40a71ff8e4b45b25fb3e49/

So something is broken in stable Chrome. Which reminds me of something I saw you and Jake were talking about on Twitter: https://twitter.com/patmeenan/status/1108059970732474368.

pmeenan commented 5 years ago

My guess is that it's something the emerging markets team is experimenting with (I think I remember something along those lines as an experiment in the loader). Kind of sucks to have another wildcard at play but hopefully it works well in the wild at-large.

pmeenan commented 5 years ago

This is the experiment I was thinking about: https://bugs.chromium.org/p/chromium/issues/detail?id=836552

Looks like it has been live for a few builds now so not sure why it is just showing up now. Maybe the link speed estimator is getting an estimate sooner than it was before.

icing commented 5 years ago

Cool! Many thanks @pmeenan! Now I get predictable waterfalls and analysing this is much easier!