Poor performance using ARGB_8888

[davemorrissey]

This issue continues the discussion from commit a3dcfbc.

By default this view uses RGB_565 to reduce memory usage and improve performance. When using ARGB_8888, under certain combinations of image size, screen size and screen density, rendering can become very slow. Simple gestures like pan and zoom cause large GPU spikes and frame rendering times in the hundreds of milliseconds even on a powerful device.

Testing with various combinations of settings has suggested the key limitation is not the total amount of image data drawn on the canvas (though this is a factor) but the size of the largest single bitmap.

RGB_565 will remain the default for now, and the view caps image density at 320dpi (retina), so most developers will not be affected by this. For those using ARGB_8888 and preferring to sacrifice image density for faithful colour reproduction, the density can be capped lower by calling setMinimumTileDpi(240). This should eliminate the problem for almost everyone, but may be noticeable on images with fine lines, e.g. maps and building plans.

Possible solutions

Render large tiles as segments

Reducing the maximum tile size to lower than the default, e.g. 2048px, causes much longer loading times but reduced rendering times. This implies that by rendering large tiles with 4 separate calls to drawBitmap for 4 segments of the bitmap, it may be possible to preserve loading performance and reduce rendering time, despite the total amount of image data written to the canvas being the same.

Convert to SurfaceView

This may help the view respond better to touch gestures but it's not clear whether it would improve rendering performance. Could be a breaking change for subclasses.

Convert to OpenGL

Converting to OpenGL would be a major change but may greatly improve performance. Could be a breaking change for subclasses.

[davemorrissey]

I have tried rendering tiles as segments, but this performed no better than default. Using a matrix to draw a part of the bitmap is no faster than drawing the whole bitmap. Any workaround for this using smaller bitmaps would make loading much slower so this is a non-starter.

On a more positive note, API 26 added Bitmap.Config.HARDWARE and this gives far better results than ARGB_8888. It's worth trying this when it's available.

[davemorrissey]

I have committed an experimental and very buggy SurfaceView implementation to the surface-view-experimental branch. Under certain circumstances this performs better than master, but in others it is slower.

SurfaceView canvas does not support hardware acceleration until API 26, and with this enabled, performance is far worse. It turns out that forcing software rendering for the view vastly improves performance for very large bitmaps that are displayed without subsampling, while quite noticeably reducing performance for smaller and subsampled bitmaps. In other words, software rendering is slower but lacks the large bitmap performance cliff edge seen with hardware rendering.

These are my results:

Bitmap config: ARGB_8888 Image: JPG photo 4912x3264 Screen: 1080x1920 landscape Device: Nexus 5X

View master

• Software canvas
  • Paging: VERY FAST
  • Pan/zoom: FAST
• Hardware canvas
  • Paging: VERY SLOW
  • Pan/zoom: VERY SLOW

SurfaceView surface-view-experimental

• Software canvas
  • Paging: MODERATE
  • Pan/zoom: MODERATE
• Hardware canvas
  • Paging: VERY FAST (after redundant invalidation fix)
  • Pan/zoom: VERY SLOW

In conclusion, it's not worth migrating to SurfaceView because better results can be achieved by calling setLayerType(LAYER_TYPE_SOFTWARE, null) on the view. For now it's up to developers to do this manually if they're displaying large bitmaps and want them displayed at the highest possible resolution.

If I can work out where the performance crossover is, this could be done automatically, although that could cause unexpected rendering behaviour in subclasses.

OpenGL is still an option.

[franciscofranco]

There's some great information about Bitmap.Config.HARDWARE here https://twitter.com/commonsguy/status/894983637585858560

[davemorrissey]

Seems like it would risky to adopt that as a solution, given switching to software rendering produces a decent result and works on the other 99.7% of devices. It does make me wonder what will happen if I try to load lots of tiles into OpenGL textures and hit a limit.

@franciscofranco I know you're using 240dpi now, but if you get the chance could you try forcing software rendering with a much higher DPI to see if you get the same result I did?

[franciscofranco]

Sure, I'll try that right now!

Software based rendering is even worse with the default 320dpi, it's barely unusable while using 8888 on the bitmap decoder and region decoder. Tested on my Pixel 2016. I can try other devices as well if that helps.

[franciscofranco]

Also, to add more fuel to the fire, setting both bitmap decoder and region bitmap decoder to 8888 makes things even worse, even something as low as 160dpi there's some stutter when scaling up/down during pinch to zoom or double tap to zoom.

[davemorrissey]

This is the opposite behaviour to what I've observed. Calling setLayerType(LAYER_TYPE_SOFTWARE, null) on the view definitely makes it perform a lot better with a large unsampled bitmap at >320dpi, and there's some stutter but it's not bad considering my test image is about 60Mb as an 8888 bitmap.

It's hard to see how a large image wouldn't be subsampled if you set the quality to 160dpi, and I've certainly never seen poor performance with a subsampled image. Enabling debug will show you if the image has been subsampled - it will have a magenta border and the "ISS [x]" label.

As you have a Nexus 5X it's worth trying that in case this is hardware dependent.

[franciscofranco]

I'll re-test everything on the Nexus 5X. With setLayerType(LAYER_TYPE_SOFTWARE, null) performance was destroyed on a Pixel. Even tested it with the native display dpi. Anyway I'll keep you posted of my tests, I want to help with this as much as I can.

[franciscofranco]

On my N5X with the default hardware layer, 320dpi and 8888 there's the usual horizontal viewpager scroll jank, but zooming in/out + panning is 10x smoother. With a software layer, same setup it's smoother (though not perfect) horizontal viewpaget scroll, but zooming in/out + pan is completely destroyed. It seems software rendering is way better on the 5X compared to the Pixel in this specific case. 160dpi definitely subsamples, and there's still some jank, but it's tolerable, on the Pixel is destroyed. So with the 5X I can confirm your observations.

[davemorrissey]

If pan+zoom are very slow with N5X/8888/>320dpi/software but fast with hardware, that's the opposite of my observations.

OpenGL may be the last hope for a universal fix. I don't think I'll have time to attempt it for a while, there are other projects I need to spend my time on.

[KnIfER]

Like game, It's vital to keep a low draw count.

When hasMissingTiles is true, ssiv will draw a lot, so I think I'd better develop a simple algorithm to avoid re-drawing overlapping bitmaps.

And what about a dynamic minimumTileDpi? For example when scale>0.5, simply calculate scale = (220/averageDpi) * scale; instead of scale = (minimumTileDpi/averageDpi) * scale;, that will make the layer of smallest sample size apear in lager scale level.

By the way RGB_565 is slower than RGB_8888 in my device although it does save memory.