search

ScottMansfield / widow

Distributed, asynchronous web crawler
GNU Lesser General Public License v2.1
26 stars 4 forks source link

Investigate more accurate timing of website response #5

Open ScottMansfield opened 9 years ago

ScottMansfield commented 9 years ago

See: com.widowcrawler.fetch.FetchWorker

Can we get better timing about the request at a lower level? The timing should be as close to on-the-wire time as possible.

ScottMansfield commented 9 years ago

I may need to consider a TTFB and whole-message response time as separate metrics. I think the current response time is neither, and I'm unsure of the timing it actually produces. It's nanos calls around Invocation.invoke()

https://jax-rs-spec.java.net/nonav/2.0-rev-a/apidocs/javax/ws/rs/client/Invocation.html#invoke()