search

taikoo / mobitest-agent

Automatically exported from code.google.com/p/mobitest-agent
0 stars 0 forks source link

Optimize iOS screen capture #10

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
See if we can make the iOS screenshots a bit faster, to support more frequent 
screenshots for fast-loading pages.

Note: the default "bz-fps" is 1 and the "Settings.bundle/Root.plist" only 
allows the developer to chose from 1, 2, 5, 10; not sub-second resolutions.  If 
we can optimize the screenshots then we should allow smaller values.

The raw screenshot work is in "UIGetScreenImage()".  We likely can't do 
anything to optimize that.

However, the post-capture work could perhaps be optimized, assuming that it's 
significant relative to the screenshot time.  The "TODO" code comments in 
BZWebViewController.mm captureScreen suggest some ideas:

1) Instead of converting the image to a JPEG via:
       NSData *imageData = UIImageJPEGRepresentation(image, 1.0);

  a) Use at a lower quality than 1.0, especially because the BZModel+ZIP.m upload will scale down these video images anyways based on the "bz-vid-jpg-quality" setting, which defaults to 0.3  (0.7 for non-video "bz-chkpoint-jpg-quality" checkpoint screenshots).

  b) Similarly, scale the images here instead of in BZModel+ZIP.m, 'though the default "bz-jpg-size" scale factor is 1.0 (i.e. don't resize).

  c) Do this conversion later

2) Instead of writing to the disk after every capture, keep them in memory and 
batch-flush as necessary (e.g. at end of each run),

Original issue reported on code.google.com by wrig...@google.com on 26 Nov 2012 at 5:13

GoogleCodeExporter commented 9 years ago 
Oops, I mean fps is 1, 2, 5, 10, so we already support 10 frames per second!  
So ignore that part...

Original comment by wrig...@google.com on 26 Nov 2012 at 5:16

GoogleCodeExporter commented 9 years ago 
Another option to try out is to not encode the images at all (and keep them in 
memory in raw form) until the end of the test.  This is actually exactly how 
the desktop agents work.

Granted, this will increase the amount of RAM the agent uses and make it more 
likely to crash from memory pressure but from what I've read the actual capture 
can be very fast and it may be the encoding/writing that is expensive.

If we capture screen shots only when the screen actually changed then it can 
mitigate the memory issues.

If an image resize is fast then resizing the capture to 1/4 resolution may also 
help with the memory pressure.

Original comment by pmee...@chromium.org on 26 Nov 2012 at 5:21

GoogleCodeExporter commented 9 years ago 
Optimizing the speed of capturing screenshots is definitely a worthwhile 
activity. 
In a (somewhat old) test we did, even capturing screenshots once a second added 
~30% to the load time on average, alongside not being granular enough.

Note that it's possible this is simply a limitation, as the resources required 
to capture a screenshot may be a lot compared to a mobile device's 
capabilities. 

Another suggestion: recent iOS versions have a build in Screen Capture 
capability (pressing the home and power button at once), which - I believe - 
was not there when we first created Mobitest. If we're lucky, maybe that 
capture is GPU accelerated, and maybe we can find some more official API to 
capture it? 

Lastly, if we can find a video recording capability on the device, that'll be 
another option.

Original comment by guy...@gmail.com on 26 Nov 2012 at 5:23