sdslnmd / concurrentlinkedhashmap

Automatically exported from code.google.com/p/concurrentlinkedhashmap
Apache License 2.0
0 stars 0 forks source link

Make capacity and weightedSize long #31

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago
I'm using CLHM for a cache, with weights denoting the sizes (in bytes) of the 
respective elements. Problem is, caches nowadays can easily surpass the 2GB 
limit of a positive int.

A workaround would be to shave a few bits off the element size for the weight, 
simply making the cache a little too eager to evict (as element sizes would be 
essentially rounded up), but making capacity and weightedSize long is a minor 
change.

Original issue reported on code.google.com by ron.pres...@gmail.com on 23 Feb 2012 at 2:42

GoogleCodeExporter commented 9 years ago
The original rational for ints was:
 - Less space for the common-case
 - Larger capacity if the base unit is changed (1 unit = 1 kb)
 - I didn't realize how popular byte weighing (e.g. byte-buffers) would be

I'm torn only because it seems byte weighing is more popular than collection 
weighing, so it might make sense. It could be too wasteful, so I wouldn't make 
the change lightly.

Guava is upstream so I'd like Charles to weigh in.

Original comment by Ben.Manes@gmail.com on 23 Feb 2012 at 10:06

GoogleCodeExporter commented 9 years ago
The problem with long weights is that it allows overflow of the maximum weight 
value. In order to avoid that you have to leave some of the weight bits unused. 
Forcing the weights to be ints seems like a perfectly sensible way to leave 
that decision in the hands of the developer. If each cache entry is less than 
2GB in size it sounds like you can easily convert to an int without loss of 
precision.

Original comment by fry@google.com on 23 Feb 2012 at 10:23

GoogleCodeExporter commented 9 years ago
Yeah, I don't think the weights should be long; int is fine (b/c, as you said, 
forcing entries below 2GB, if byte weights are used is hardly a restriction) - 
only the capacity (and weightedSize) should be long.

Original comment by ron.pres...@gmail.com on 23 Feb 2012 at 10:28

GoogleCodeExporter commented 9 years ago
In CLHM I detect and fail on an overflow, so the max weight isn't really 
Integer.MAX_VALUE. I left Google before you finished adding weight support to 
CacheBuilder. Is there a check? If not, you might want to grab the test case 
from WeighersTest.

Original comment by Ben.Manes@gmail.com on 23 Feb 2012 at 10:29

GoogleCodeExporter commented 9 years ago
Oh, I see. Sorry I was confused since previous requests always asked for long 
weights.

This sounds fair to me, but I haven't thought about it through yet.

Original comment by Ben.Manes@gmail.com on 23 Feb 2012 at 10:33

GoogleCodeExporter commented 9 years ago
Fixed in v1.3. I plan on releasing this tonight.

Also introduced EntryWeigher<K, V> to allow key/value weighing. We fixed this 
oversight in Guava's CacheBuilder from the get-go.

Original comment by Ben.Manes@gmail.com on 8 May 2012 at 9:43