Extending and adding a interceptor to give an output similar to default chrome network request

[sybarite]

I am using the okhttp interceptor provided by stetho, but I am only getting latency and download times. How do I provision okhttp to get a network timeline output like above? It would be great if a library developer would point to right direction to develop such a functionality if its not present

[jasta]

Unfortunately we currently don't support collecting or sending this data. These are actually two separate problems that have to be solved separately:

1. NetworkEventReporter (and ...Impl.java) would have to offer an expanded API to collect this information and then relay that along the Chrome DevTools protocol. This part should actually be pretty trivial, just some minor improvements to the API needed.
2. The helper modules, stetho-urlconnection and stetho-okhttp would need to be updated to take advantage of this, providing whatever telemetry is possible with this libraries. Unfortunately, and the main reason this hasn't been done yet, I don't believe okhttp itself actually offers most of this information so it would effectively be impossible to prove for the vast majority of our users.

I have long since felt okhttp lacking in its accounting/statistics capabilities so this might be high time to put some pressure on them to open up (or maybe just document?) APIs.

I know this sounds contrived, but patches and some research legwork would be greatly appreciated here. I too would love to see this feature.

[sybarite]

@jasta Thanks a lot for the information though. If okhttp would need some additional exposure of their API or better documentation its better I start from there and get to modifying the NetworkEventReporter later. First step is to know all the information we need to pass to the Chrome DevTools protocol is present and can be collected.

[jasta]

@sybarite agreed, that approach sounds right. To give you a sense of what the DevTools protocol lets you do, see https://github.com/facebook/stetho/blob/master/build-tools/protocol.json. This is the "documentation" for the DevTools protocol and explains what kinds of messages we can send/receive. In this case, the ResourceTiming object is not even sent at all by Stetho today, but obviously if we start the data would show up in the UI :)

FWIW, we can currently populate a few of these fields today we just didn't. In particular, that's sendStart, sendEnd, and receiveHeadersEnd.

[sybarite]

@jasta Oh nice thanks for the links! Also I have several doubts now that I am stepping into one class at a time. I understand the the NetworkEventReporter is a singleton. For domain event the request and response information is expected to be passed in the InspectorRequest and InspectorResponse objects.

And the ResourceTiming object is expected in the Response object for chrome dev tools protocol, but since in the InspectorResponse object there is no context for the Request, I am not sure how I will add all the request timings to the ResourceTiming object.

Basically I need these timing data to persist between these calls and finally attach it to Response object expected in the chrome dev tools protocol. Or maybe I am not understanding the current architecture correctly, it would be fantastic if you could give me an overview of the architecture on collecting the Network domain information.

Also I am trying to get all the timing information for the stetho-urlconnection library first just to understand how I will go about collecting the timing. I will start grokking okhttp later for the same functionality.

[jasta]

It is possible to share information across request and response in the current design by keying off the request id. We do this currently actually because the protocol decouples the response headers and response body. The latter we store in a file as it may be very large and will not necessarily be accessed by the DevTools (only when you click on it is it requested).

For your purposes, I suspect the easiest thing to do would be to simply use a SparseArray to track the timing information, keyed by requestId which is common to both InspectorRequest and InspectorResponse. The data structure would be added to in requestWillBeSent and removed from in responseHeadersReceived.

I recommend associating this data structure with NetworkPeerManager so that that onLastPeerUnregistered method can be used to empty the data structure to prevent leaks. This might be a good place even to print a debug statement for each entry remaining in the list, indicating some kind of programmer error in failing to call responseHeadersReceived.

[bangarharshit]

This is released in okhttp 3.9 Eventlistener.

While I am able to get all the timing such as DNS, SSL (no proxy callback), it is not meant to access response (their documentation says - Extend this class to monitor the quantity, size, and duration of). I think an experimental StethoEventListener can co-exist with the interceptor (it can be used for resource timing). Some of the callbacks in listener are called multiple times for redirect so we may need to filter 3xx.

Also, updated ResourceTiming has new fields on service workers and push start (I believe these things can be set to 0 as can be in this link).

I can send a quick PR by this weekend.

[jasta]

A PR would be awesome! I'd love to see this in Stetho, but we will likely need to upgrade to the latest dev tools to get all of the benefit. Ideally we'd start with getting what we can in current dev tools and then look at what it would take to upgrade to the latest (we're years behind now, sadly).

[NormanOu]

Hello, I am trying to add Network.ResourceTiming into Network.Reponse, to support displaying detail request information on chrome. But I meet a problem. Every time I set any data to ResourceTiming, the request shown on chrome will not 'end' any more(it seems like the latter event 'Network.dataReceived' and 'Network.loadingFinished' is not handled anymore. Has anybody tried this? Any help would be nice