Closed GoogleCodeExporter closed 9 years ago
Relatedly, it'd be good to test these things in reproducible network
environments (e.g. on a clean 1Gbps network, what do we see?. On a network with
1% loss, what do we see? On a machine with 30ms RTT, what do we see?)
Original comment by AaronMat...@gmail.com
on 13 Mar 2015 at 2:55
Would following test plan be good?
Measure performance results depending on:
- network type (10mb/100mb/1gb),
- packet loss (e.g. starting from 1% to 10% with step=1%)
- delay value: (e.g. 10ms/30ms/100ms/300ms/500ms/1s/2s)
mixing these things so test would look as follows:
1. Network: 100mb, packet loss: 1%, RTT: 30ms
2. Network: 100mb, packet loss: 2%, RTT: 100ms
etc
I can test mentioned things on my machine with Linux system and Firefox/Chrome
browsers using newest flash version (on linux it is quite old anyway:
11.2.202.451) and java 7u76.
What do you think about such tests? Is it what you had in mind or should it be
performed another way (any additional notes would be then very appreciated)?
Original comment by skost...@soldevelo.com
on 19 Mar 2015 at 10:23
Original comment by skost...@soldevelo.com
on 19 Mar 2015 at 12:31
That's pretty much what I was thinking. You could use netem on a linux host
acting as a router to create a reproducible network setup with various
performance characteristics.
As to the end host, do you have access to a Windows host or a Mac? Those are
the stats that are probably more interesting to the user base.
Original comment by AaronMat...@gmail.com
on 19 Mar 2015 at 5:15
Unfortunately I don't have access to device with Mac system but I can test it
using Windows instead then.
Original comment by skost...@soldevelo.com
on 20 Mar 2015 at 8:36
Attaching test results. Below short information how tests were performed:
System: Windows 8.1
Browsers: Firefox 35.0.1, Chrome 41.0.2272.101
Java version: 8u40
Flash version: 17.0.0.134
In attached file there are 3 sections: first is with results from Firefox and
comparing flash/java performance, second the same for Chrome and third is
comparison of results obtained in these two browsers. All results are average
number of 5 tests done in the same environment. Also I have not tested in
detail for 10mb network as it's rather obsolete today and also not tested
further for remaining delay/packet loss values as download values in results
for such configuration were too low to mention (few or below 1 mb/s).
Configuration was changed using netem on Linux machine which performed as
web10g server using newest code from trunk r1217 (Ubuntu 14.04 LTS with 3.9.1
kernel and 2.0.7 userland) which I directly connected with second machine with
Windows.
Do these results satisfy requirements mentioned in ticket description or
something should be added/changed ?
Original comment by skost...@soldevelo.com
on 23 Mar 2015 at 9:44
Attachments:
Could you include the standard deviation of the 5 tests? These numbers are all
over the place...
Original comment by AaronMat...@gmail.com
on 23 Mar 2015 at 2:30
It turned out that for some configurations deviation was too high so I
performed additional tests and now for all configurations its minimum 10
(instead of 5), I have performed as many attempts as were needed to reduce
relative standard deviation below 10%. Results show that the biggest difference
is between flash version integrated with Chrome and the one which comes from
Adobe (~50%). Also upload values differences for 0% packet loss and 0ms delay
is quite high. All details are visible in spreadsheet where you can define
thresholds and see cells automatically being colored indicating which ones
match defined requirements. Is that ok now or something should be
added/changed?
Original comment by skost...@soldevelo.com
on 25 Mar 2015 at 2:44
Attachments:
This all looks good. Thanks! Would it be possible to get some Linux numbers too?
Original comment by AaronMat...@gmail.com
on 25 Mar 2015 at 3:25
New spreadsheet with Linux system results added (tested on Java and Flash
plugins for Firefox and Flash only for Chrome as Java support has been removed
since version 35). Results show that there is huge difference between very old
version of Flash in Linux's Firefox (11.x) when compared to results from newest
one from Windows (17.x). Also results overall differ quite a lot between both
systems. Are these numbers enough or is something missing?
Original comment by skost...@soldevelo.com
on 27 Mar 2015 at 1:43
Attachments:
After some investigation it turned out that for 100mb/s network speed results
between Windows and Linux differed so much because of duplex config, for Linux
it was full-duplex while for Windows half. I then configured Windows to work in
full-duplex mode too and attached file contains updated results for 100mb/s for
Windows.
Original comment by skost...@soldevelo.com
on 30 Mar 2015 at 1:51
Attachments:
These look great. Thanks! Would it be possible to include some warnings in the
web client if Linux + Flash, or Chrome + Flash get used?
Original comment by AaronMat...@gmail.com
on 31 Mar 2015 at 12:12
I've added proper changes on web-client-browser-os-warnings branch. If they
look ok for you I could merge them into trunk.
Original comment by skost...@soldevelo.com
on 31 Mar 2015 at 1:34
Looks good. Closing.
Original comment by AaronMat...@gmail.com
on 31 Mar 2015 at 3:02
Original issue reported on code.google.com by
AaronMat...@gmail.com
on 13 Mar 2015 at 2:50