Quantify (and remove) latency due to client side stack

[mrdavidlaing]

The client side network stack is as follows:

1. LatencyMonitor <- captured in the Application=CiapiLatencyCollector measurement
2. CIAPI.CS <- captured in the Application=CiapiLatencyCollector.AllServiceMonitor.BuiltIn measurement
3. .NET
4. Windows
5. Amazon VM network stack

It appears that about 10% of the measured request latency is due to latency in the first 2 parts of the client side measurement stack:

Specifically, in comparing 120 Trade measurements from 2012-08-08 09:20 - 1h

 App                                                Location        FunctionName                                                Count   ExceptionsCount Average     Min         Max
 ===========================================================================================================================================================================================
 CiapiLatencyCollector                              Ireland//(All)  CIAPI.LogIn                                                 61      0               0.410149411 0.2749149   3.2537905
 CiapiLatencyCollector.AllServiceMonitor.BuiltIn    Ireland//(All)  https://ciapiprepr......co.uk/TradingApi/session            60      0               0.354466667 0.273       3.239
                                                                                                                                    ========================================================
                                                                                                                                        Diff            0.055682744 0.0019149   0.0147905

 CiapiLatencyCollector                              Ireland//(All)  CIAPI.Trade                                                 122     0               0.39847638  0.2871016   1.6627133
 CiapiLatencyCollector.AllServiceMonitor.BuiltIn    Ireland//(All)  https://ciapiprepr....co.uk/TradingApi/order/newtradeorder  119     0               0.352756303 0.283       1.615
                                                                                                                                    ========================================================
                                                                                                                                        Diff            0.045720077 0.0041016   0.0477133

we see a average different of 0.04 to 0.05 seconds between measurements of the same request taken at level 1 and level 2 of the stack.

What causes this difference? Can we remove it?

Which measurement is more correct, and which should we be using?

[mrdavidlaing]

See related CIAPI.CS issue https://github.com/cityindex/CIAPI.CS/issues/187

[fandrei]

I found this in Salient.ReliableHttpClient: https://github.com/bitpusher/Salient.ReliableHttpClient/blob/master/src/Salient.ReliableHttpClient/RequestController.cs#L46 https://github.com/bitpusher/Salient.ReliableHttpClient/blob/master/src/Salient.ReliableHttpClient/RequestController.cs#L129 This code means that waiting period of 0.05 seconds is taken after sending each burst of CIAPI requests. As an experiment, I changed this value to 0.001, and diff between internal and external latency reduced to level under 1%

[mrdavidlaing]

So, do we:

1. Remove the latency in CIAPI.CS and keep measuring outside
2. Switch measurements to data collected inside CIAPI.CS, which doesn't (?) record this delay

[fandrei]

I think it's better to set delay to much smaller value (~0.001 seconds), because even if we can remove it from our reports, it still does affect experience of CIAPI.CS users. And it's better to keep both internal and external measurements, so we can see if there are any other disrepancies.

[fandrei]

I've made several tests on my PC with different waiting period: https://github.com/fandrei/CiapiLatencyTest

0.001 CPU time used, seconds: 0.703125 total time: 98.3340983 CPU time used, seconds: 0.65625 total time: 114.9099609 CPU time used, seconds: 0.8125 total time: 99.3258835

0.05 CPU time used, seconds: 0.796875 total time: 102.7563944 CPU time used, seconds: 0.875 total time: 102.0175365 CPU time used, seconds: 0.921875 total time: 97.4349284

There are no any signs that CIAPI.CS with waiting period 0.001 consumes more CPU.

Not sure how to measure performance on the phone, as I don't have any WP7 phones :)

[mrdavidlaing]

Nice!

Then I propose that we just reduce the waiting period down to 1 ms.

Sky - can you confirm whether this makes any difference on the Win7 Phone

D

On 16 August 2012 08:15, Andrei Faber notifications@github.com wrote:

I've made several tests on my PC with different waiting period: https://github.com/fandrei/CiapiLatencyTest

0.001 CPU time used, seconds: 0.703125 total time: 98.3340983 CPU time used, seconds: 0.65625 total time: 114.9099609 CPU time used, seconds: 0.8125 total time: 99.3258835

0.05 CPU time used, seconds: 0.796875 total time: 102.7563944 CPU time used, seconds: 0.875 total time: 102.0175365 CPU time used, seconds: 0.921875 total time: 97.4349284

There are no any signs that CIAPI.CS with waiting period 0.001 consumes more CPU.

Not sure how to measure performance on the phone, as I don't have any WP7 phones :)

— Reply to this email directly or view it on GitHubhttps://github.com/fandrei/AppMetrics/issues/101#issuecomment-7778368.

David Laing Open source @ City Index - github.com/cityindex http://davidlaing.com Twitter: @davidlaing

[mrdavidlaing]

@Andrei - Can you ensure that AppMetrics is using CIAPI.CS > v8.2 & thus ReliableHttpClient > 1.0.6.17, and that the CIAPI Latency Collectors:

1. Do not still introduce an extra 50ms latency
2. Are taking measurements with the Stopwatch()

If both of those are true, then this ticket can be closed.