WPT cookies/http_state tests are slow

[pshaughn]

Running on a machine that's much slower than taskcluster, I find that I have to set --timeout-multiplier as high as 4 to get through some tests consistently. This problem is not necessarily about cookies, as the tests are doing entire HTTP roundtrips against the cookie-setter.py.

If it is actually the cookies slowing us down, it may be that the cookie jar is getting deserialized and re-serialized too often; I get the impression from the code that we are passing around the entire cookie store even when we already know which one URL we want to look up.

[jdm]

@cybai and I did some more investigation - the domain_state tests have test cases that include a Location header but no 3xx redirect code. Our fetch code (and the spec) handles this by ignoring the header, but the code in the constellation for handling redirects for navigations only looks for the presence of the header and gets confused: https://github.com/servo/servo/blob/master/components/constellation/network_listener.rs

[CYBAI]

26475 will fix most of TIMEOUT tests but there's still a TIMEOUT test case in chromium-tests.html. Will investigate that one.

[atouchet]

Is this supposed to be closed?

[CYBAI]

@atouchet Thanks! This gets better now but I will expect it's closed when we fix the chromium-tests.html timeout! Let me reopen it!

[CYBAI]

Just did some investigation. The last TIMEOUT test is disabled-chromium0022 in chromium-tests.html.

While running the page directly with

$ RUST_LOG=net ./mach run http://wpt.live/cookies/http-state/resources/cookie-setter.py\?file\=disabled-chromium0022

then, I will see above error page without any response log because we've failed to request.

[2020-05-09T11:14:43Z DEBUG net::image_cache] New image cache
[2020-05-09T11:14:46Z INFO  net::cookie_storage]  === COOKIES SENT:
[2020-05-09T11:14:46Z DEBUG net::http_cache] trying to construct cache response for "http://wpt.live/cookies/http-state/resou... (8400db339d2aa31a)"
[2020-05-09T11:14:46Z INFO  net::http_loader] GET request for http://wpt.live/cookies/http-state/resources/cookie-setter.py?file=disabled-chromium0022
[2020-05-09T11:14:46Z INFO  net::http_loader]  - ("accept", "text/html, application/xhtml+xml, application/xml; q=0.9, */*; q=0.8")
[2020-05-09T11:14:46Z INFO  net::http_loader]  - ("accept-language", "en-US, en; q=0.5")
[2020-05-09T11:14:46Z INFO  net::http_loader]  - ("user-agent", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:75.0) Servo/1.0 Firefox/75.0")
[2020-05-09T11:14:46Z INFO  net::http_loader]  - ("accept-encoding", "gzip, deflate, br")

Due to the error message and also the test case contains a new-line character, I suspect if this is an issue related to hyper (maybe hyperium/hyper#2083 ?)

https://github.com/servo/servo/blob/536c60afe197a98a836b1a5740ac0b25210dbd6c/tests/wpt/web-platform-tests/cookies/http-state/resources/test-files/disabled-chromium0022-test#L1

But, with more investigation to spec (whatwg/xhr#165) and wpt tests (web-platform-tests/wpt#13663, web-platform-tests/wpt#10424), looks like it's correct to reject the request when a header contains a 0x00 value 🤔

Also, we don't TIMEOUT in the fetch/api/basic/header-value-null-byte.any.js test.

I wonder if it's related to how we treat network error for navigate request instead 🤔 ?

[pshaughn]

From what I'm remembering about the issue I originally saw, it wasn't just that some tests were getting an inevitable TIMEOUT, it was also that there were just plain slow ones that'd time out for me under usual CPU load but could eventually pass if allowed enough extra time to do it. What you're talking about here is mostly what I had posted as #25604.

[CYBAI]

@pshaughn oh, thanks! and what I investigated at https://github.com/servo/servo/issues/25606#issuecomment-626165238 is basically #25605.