pmeenan
commented
2 years ago I have the initial POC working of streaming the netlog and capturing netlogs for each individual step on Linux (have to test MacOS but it should use the same flow). I'm working on porting the trace netlog event parsing over and then will submit a PR but wanted to get an issue in place as a heads-up.
It looks like the netlog captures some events and data that we don't get from the trace-based events as well so we'll be able to log much better details for QUIC, HTTP/2, certificates, etc.
The current plan is to keep the trace-based netlogs as a fallback for cases where the file-based streaming netlogs won't work (Android devices in particular).
The WPT agent usually uses the netlog trace events to get the underlying "truth" for the request data but that source of data can sometimes fail (and isn't the main netlog that Chrome uses for diagnosing issues). It required forcing the in-process Network Service feature but even then we are seeing cases in Chrome 102 where sometimes the netlog events are not being captured in the trace.
The agent falls back to dev tools events when the netlog isn't available but the timing of the requests can be completely off (HTTP/2 push) or the requests can show as if they are coming from cache (103 early hints) which causes them to not be displayed.
Historically we have relied on the trace events because the file-based netlog doesn't flush to disk until the browser closes, making it impossible to capture just the parts of the netlog that are relevant to a given step.
It turns out that using named pipes (fifo's) on Linux and MacOS (and possibly //.PIPE/xxx named pipes on Windows) allows reading of the first-class netlog event stream in realtime, enabling support for capturing just the events relevant to a given step and to process them in realtime as they are happening. This will significantly increase the reliability of the netlog events and allow for running Chrome without special features.