Implement Caffine Cache Provider

wezell commented 8 years ago

Caffine cache ( https://github.com/ben-manes/caffeine ) is the new hotness with regards to non-blocking caches. It is almost a drop in replacement for Google Guava. I wanted to see the performance difference between it and guava.

Expected Behavior

Caffine Cache should perform better

Current Behavior

Guava is good, but I've been benchmarking static files and see most threads stuck in guava, e.g.


"url:GET//demo.dotcms.com/ | lang:1 | ip:127.0.0.1 | Admin:false | start:09-21-2016 08:19:17 EDT {/Users/will/git/tomcat8/webapps/ROOT/assets/f/d/fd370dbf-42c0-4f8e-82ae-c63fa325c23e/fileAsset/header.vtl}" #362 daemon prio=5 os_prio=31 tid=0x00007f9b85aee000 nid=0x2b103 runnable [0x0000700014862000]
   java.lang.Thread.State: RUNNABLE
    at com.dotcms.repackage.com.google.common.cache.LocalCache$Segment.getAndRecordStats(LocalCache.java:2383)
    at com.dotcms.repackage.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2351)
    at com.dotcms.repackage.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313)
    - locked <0x00000007928d1998> (a com.dotcms.repackage.com.google.common.cache.LocalCache$StrongAccessEntry)
    at com.dotcms.repackage.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228)
    at com.dotcms.repackage.com.google.common.cache.LocalCache.get(LocalCache.java:3965)
    at com.dotcms.repackage.com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4764)
    at com.dotmarketing.business.cache.provider.guava.GuavaCache.get(GuavaCache.java:98)
    at com.dotcms.enterprise.cache.provider.CacheProviderAPIImpl.get(CacheProviderAPIImpl.java:341)
    at com.dotmarketing.business.ChainableCacheAdministratorImpl.get(ChainableCacheAdministratorImpl.java:312)
    at com.dotmarketing.business.CacheLocator$CommitListenerCacheWrapper.get(CacheLocator.java:97)
    at com.dotmarketing.business.IdentifierCacheImpl.getIdentifierFromInode(IdentifierCacheImpl.java:130)
    at com.dotmarketing.business.IdentifierFactoryImpl.find(IdentifierFactoryImpl.java:255)

Possible Solution

Implement a caffine cache

Known Workarounds

Steps to Reproduce (for bugs)

ab -n 100000 -c 25 http://127.0.0.1:8080/contentAsset/image/97e2e928-8f6c-4253-a07e-2eb1d5d10e3f/image/

Guava cache

hot-fries:dotcms3 will$ ab -n 100000 -c 25 http://127.0.0.1:8080/contentAsset/image/97e2e928-8f6c-4253-a07e-2eb1d5d10e3f/image/
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests

Server Software:        Apache-Coyote/1.1
Server Hostname:        127.0.0.1
Server Port:            8080

Document Path:          /contentAsset/image/97e2e928-8f6c-4253-a07e-2eb1d5d10e3f/image/
Document Length:        36284 bytes

Concurrency Level:      25
Time taken for tests:   70.771 seconds
Complete requests:      100000
Failed requests:        0
Total transferred:      3668800000 bytes
HTML transferred:       3628400000 bytes
Requests per second:    1413.00 [#/sec] (mean)
Time per request:       17.693 [ms] (mean)
Time per request:       0.708 [ms] (mean, across all concurrent requests)
Transfer rate:          50625.15 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.8      0     111
Processing:     2   17  15.5     15     761
Waiting:        2   17  15.5     15     761
Total:          3   18  15.5     15     762

Percentage of the requests served within a certain time (ms)
  50%     15
  66%     16
  75%     16
  80%     17
  90%     21
  95%     22
  98%     24
  99%     95
 100%    762 (longest request)

Caffine Cache

hot-fries:dotcms3 will$ ab -n 100000 -c 25 http://127.0.0.1:8080/contentAsset/image/97e2e928-8f6c-4253-a07e-2eb1d5d10e3f/image/
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests

Server Software:        Apache-Coyote/1.1
Server Hostname:        127.0.0.1
Server Port:            8080

Document Path:          /contentAsset/image/97e2e928-8f6c-4253-a07e-2eb1d5d10e3f/image/
Document Length:        36284 bytes

Concurrency Level:      25
Time taken for tests:   29.950 seconds
Complete requests:      100000
Failed requests:        0
Total transferred:      3668800000 bytes
HTML transferred:       3628400000 bytes
Requests per second:    3338.85 [#/sec] (mean)
Time per request:       7.488 [ms] (mean)
Time per request:       0.300 [ms] (mean, across all concurrent requests)
Transfer rate:          119624.66 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    2  15.8      1    1262
Processing:     1    6  40.0      4    1618
Waiting:        0    4  28.4      3    1616
Total:          1    7  43.0      5    1619

Percentage of the requests served within a certain time (ms)
  50%      5
  66%      5
  75%      6
  80%      6
  90%      7
  95%      8
  98%     10
  99%     13
 100%   1619 (longest request)

Your Environment

OSX iMac 4 core i7 running in Eclipse debug mode

wezell commented 8 years ago

More stats - this is rendering the home page without the page cache

Guava

hot-fries:dotcms3 will$ ab -n 10000 -c 25 http://127.0.0.1:8080/
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
Completed 9000 requests
Completed 10000 requests
Finished 10000 requests

Server Software:        Apache-Coyote/1.1
Server Hostname:        127.0.0.1
Server Port:            8080

Document Path:          /
Document Length:        33113 bytes

Concurrency Level:      25
Time taken for tests:   54.959 seconds
Complete requests:      10000
Failed requests:        0
Total transferred:      335730000 bytes
HTML transferred:       331130000 bytes
Requests per second:    181.95 [#/sec] (mean)
Time per request:       137.397 [ms] (mean)
Time per request:       5.496 [ms] (mean, across all concurrent requests)
Transfer rate:          5965.59 [Kbytes/sec] received

Caffine

hot-fries:dotcms3 will$ ab -n 10000 -c 25 http://127.0.0.1:8080/
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
Completed 9000 requests
Completed 10000 requests
Finished 10000 requests

Server Software:        Apache-Coyote/1.1
Server Hostname:        127.0.0.1
Server Port:            8080

Document Path:          /
Document Length:        33113 bytes

Concurrency Level:      25
Time taken for tests:   50.523 seconds
Complete requests:      10000
Failed requests:        0
Total transferred:      335730000 bytes
HTML transferred:       331130000 bytes
Requests per second:    197.93 [#/sec] (mean)
Time per request:       126.308 [ms] (mean)
Time per request:       5.052 [ms] (mean, across all concurrent requests)
Transfer rate:          6489.33 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.2      0       3
Processing:    26  126  46.8    118     613
Waiting:       25  126  46.7    118     613
Total:         26  126  46.8    118     613

Percentage of the requests served within a certain time (ms)
  50%    118
  66%    133
  75%    144
  80%    151
  90%    175
  95%    198
  98%    235
  99%    271
 100%    613 (longest request)

jgambarios commented 8 years ago

PR: https://github.com/dotCMS/core/pull/9819

john-thomas-dotcms commented 7 years ago

New doc for Caffeine cache provider created: http://auth.dotcms.com/docs/latest/caffeine-cache-provider

Note that the version for this doc is currently marked as 4.0; if the Caffeine cache provider is moved into release 3.7.0, please re-mark this issue as "Doc: Needs Doc" so we can make sure the doc version gets updated (and the doc gets pushed properly when 3.7.0 is released). Thank you!

ben-manes commented 7 years ago

Thanks @john-thomas-dotcms!

Two minor nits,

Typo in my name ;)
Caffeine doesn't have a concurrencyLevel parameter (not segmented)

Cheers!

dotCMS / core