Reduce overhead from PINCache

[Adlai-Holler]

Especially for slower devices, the file system shuffling and the metadata loading can be really expensive.

To investigate the option of using NSURLCache as a disk cache, I created a sample app based on Texture. It shows 1000 images and you can autoscroll to the bottom on a timer.

In summary, total CPU time of the app on my iPhone 7, verified across multiple runs amounted to:

Total CPU time:
PINCache, cold: 58.25s
PINCache, warm: 58.77s
URLCache, cold: 52.14s
URLCache, warm: 48.75s

Filtered CPU time

Searching for "PINDiskCache":
PINCache, cold: 5.25s
PINCache, warm: 5.37s

Searching for "CFURLCache", since searching for NSURLCache gets you few results due to asynchronous architecture of CFURLCache:
URLCache, cold: 1.78s
URLCache, warm: 1.65s

My internet connection was fast enough that progressive images were not rendered, but a more fair test would turn it off explicitly. There's other overhead inside PINRemoteImage too, due to its support for concurrent requests to the same URL, and GIF and stuff, but even still this difference is whopping.

Video, trace files, and source code are here: https://www.dropbox.com/s/2ay0mgtg8cm5pq0/PINCache%20vs%20URLCache.zip

Info about NSURLCache

• It's a wrapper for CFURLCache (private, C++ based) and it uses a sqlite3 database under the hood. It generates an internal key for each URL request.
• It doesn't support "read from memory cache only."
• Its memory cache doesn't store UIImage objects, only NSDatas so we may have to keep some kind of NSData -> UIImage map to get the most out of the memory cache since UIImage creation and destruction are both expensive.
• It supports cache control headers so we get TTL and whatnot.

An alternative

https://github.com/ibireme/YYCache is a multilevel cache that claims to be lightning fast.

• No support for cache control.
• Open source (good).
• Not tested for years by Apple (bad).

The next step I think is to drop YYCache into my sample project and see how it performs. Performance shouldn't be the only consideration here but the cost/benefit of finding that out is pretty good here.

After that, whichever approach we choose, I think should live in a branch as a runtime option PINRemoteImage and we can target that branch. It can be hacky as long as it's well-gated and the hacks aren't so bad that we'll have to completely rewrite it to productionize it. That way we can do A/B tests in prod to see how each approach performs. cc @garrettmoon @appleguy @maicki @nguyenhuy

[Adlai-Holler]

Wow YYCache data is in.

Total CPU time:
PINCache, cold: 58.25s
PINCache, warm: 58.77s
URLCache, cold: 52.14s
URLCache, warm: 48.75s
YYCache, cold: 43.62s
YYCache, warm: 43.22s

We have a clear winner! @ibireme has been super great to us in the past and he delivers again with this awesome library.

[appleguy]

That's pretty damn impressive on YYCache!! I wonder though, if it's easy to drop in NSURLCache, that might be the way to go as it has cache-control support out of the box. Upgrading to YYCache would be a sane thing to do, though it sounds like quite a bit of extra work to implement the cache-control semantics...and indeed that might be one of the reasons for the performance delta.

[Adlai-Holler]

Agreed, the binary size and the flexibility benefits are nice. Since we have need for the cache control support I'll go that route.

[wiseoldduck]

https://github.com/pinterest/PINRemoteImage/pull/477 is a tiny change that should help with allowing one to configure a NSURLCache via NSURLSession, and, if they also want, to disable PINCache altogether.