Closed GoogleCodeExporter closed 9 years ago
Original comment by Ben.Manes@gmail.com
on 18 Feb 2009 at 8:22
I hope to do this once I'm no longer releasing a public snapshot. This may be
fairly
soon, as I finally got over the last algorithmic design hurdle. Right now its
more
of a "have fun, but use at your own risk" type of deal.
Original comment by Ben.Manes@gmail.com
on 27 Feb 2009 at 6:50
Original comment by Ben.Manes@gmail.com
on 28 Feb 2009 at 9:03
Original comment by Ben.Manes@gmail.com
on 25 Mar 2009 at 2:49
Hey, cool work on this project...
I desperately need something like your implementation for jmemcached (see
http://code.google.com/p/jmemcache-daemon). So question is...
1. How close are you to a public release?
2. How close are you to having a release in a public Maven repository?
Original comment by ryan.daum
on 21 Aug 2009 at 3:14
I just haven't had any time to work on it for the last few months. When I left
off,
my unit tests showed a race condition that I was having a hard time debug'ing.
It
was a bit nasty in that the condition was repeatable but challenging to
determine
the order of events creating it. I just haven't had any time to get back to it,
though. I could instead supply you with a tweaked version that uses a
ReentrantLock
around the link chain to ensure correctness, but have a small critical section
vs.
synchronizing a LHM.
In your case, since you have one very large cache, you either want a
soft-reference
based approach (e.g. Google Collections) to rely on GC's LRU strategy, or a
striped
LRU facade. You need LRU due to the generational approach often used with
memcached
to simulate tagged flushing, so a SecondChanceFifo would probably suffer by not
being able to tell which is really a hot entry. Simply create an array of LRU
maps
and hash to one by the key. With a good spreading function (see CHM) you'll get
close to a true LRU but split the contention across multiple locks. I can
quickly
whip that up for you, if you'd like.
Original comment by Ben.Manes@gmail.com
on 21 Aug 2009 at 4:29
Thanks for your response.
The map I need needs to provide the following characteristics:
* Likely some type of 'striped' locking as you describe in your wiki page, as much as
I understand it. Basically, to improve concurrency, it should not be necessary
to lock
the entire store for operations that affect one entry only
* The ability to perform several operations with one lock (i.e. for CAS or INCR/DECR,
a get followed by read, with a write lock for the whole operation).
* It ought to be possible to expire items not only in terms of a LRU policy, but also
when the size of the items exceeds the maximum declared cache size.
So there are a number of constraints, and I fear I'm not qualified to write the
data
structure myself. What's there now is far from efficient under heavy concurrent
threading because it relies on a global lock.
Is the last point even possible with your class & its ExpirationPolicy?
Original comment by ryan.daum
on 21 Aug 2009 at 6:08
[deleted comment]
A CAS is supported, and a increment/decrement with a get followed by an CAS
increment (e.g. AtomicLong). I doubt that it needs to be locked throughout
those
operations.
The size is something that I've wanted to add for fun, but its a non-core
feature.
Basically I would add the ability to supply a "weight" for an entry and be used
when
determining whether the capacity was exceeded. This is useful for caching
collections, which risk being unbounded lists by weight an element by its
list's
length. For memory, you would need to enable agent instrumentation (e.g. see
SizeOf
project), which will be a faster than serializing to a byte[] to get its length.
Memcached also supports time-based eviction (expiration) which you'll need to
handle. I have that through a decorator, which I should probably post on the
wiki...
If you want help we can continue the discussion in the group or email to not
divert
the discussion of this task.
Original comment by Ben.Manes@gmail.com
on 21 Aug 2009 at 7:44
O.k. I'll contact you re: this topic off-issue.
But on topic for this issue, if you want me to set up a Maven build for you, I
can do
that...
Original comment by ryan.daum
on 21 Aug 2009 at 10:19
Oh sure, that would be really great. :)
Original comment by Ben.Manes@gmail.com
on 21 Aug 2009 at 10:26
From: Ian Clarke <ian.cla...@gmail.com>
To: concurrentlinkedhashmap@googlegroups.com
Subject: Re: [CLHM] Re: Maven repository
Ben,
Might I suggest this: https://maven2-repository.dev.java.net/
This is what Google recommend that Google Code projects use.
Kind regards,
Ian.
Original comment by Ben.Manes@gmail.com
on 10 Nov 2009 at 8:54
Sonatype offers hosting & mirroring to maven central for open source projects
and they
have step-by-step instructions on how to do it:
http://nexus.sonatype.org/oss-repository-hosting.html
Original comment by ryan.daum
on 27 Nov 2009 at 3:27
Thanks for the nudge! I've been swamped with starting a new job (Google),
moving,
etc. so I've been pretty slack on this. Now that its starting to settle down,
I'll
try to do this Tues/Weds evening.
Original comment by Ben.Manes@gmail.com
on 1 Dec 2009 at 2:15
Hey Adam, can you take this one? We can publish clhm-release-1.0-lru.jar to a
maven
site and continue on with LIRS in trunk.
Thanks!
Ben
Original comment by Ben.Manes@gmail.com
on 13 Apr 2010 at 5:51
Does this library run on JDK 1.5? or JRE 1.6 is mandatory? I have requirement
to use
it on JRE 1.5, but when I compile on it it shows UnsupportedClassVersionError:
Bad
version number in .class file.
Original comment by anandw...@gmail.com
on 14 Apr 2010 at 10:08
There are no Java-6 features required. The advantages was that JDK5 is
end-of-life,
AbstractMap$SimpleEntry is public so it doesn't need to be duplicated, and
AbstractCollection#toArray()
was fixed to be threadsafe so it doesn't need to be overridden (keySet, values,
entrySet).
A backport could be provided. Please open a new issue for that task.
Original comment by Ben.Manes@gmail.com
on 14 Apr 2010 at 10:17
I followed the java.net guide and published jdk5/jdk6 versions of the v1.0-LRU
release. While the "maven deploy" succeeded, I do not see anything when
browsing the
repository. I'll check again Monday in hopes that there is a staging process
and, if
not, will need someone with Maven skills to step in.
Maven users should be able to either of the following:
<!-- JDK 6 -->
<dependency>
<groupId>com.googlecode.concurrentlinkedhashmap</groupId>
<artifactId>concurrentlinkedhashmap-lru</artifactId>
<version>1.0</version>
</dependency>
<!-- JDK 5 -->
<dependency>
<groupId>com.googlecode.concurrentlinkedhashmap</groupId>
<artifactId>concurrentlinkedhashmap-lru</artifactId>
<version>1.0_jdk5</version>
</dependency>
I followed the guide and got the following output:
$ mvn deploy
...
[INFO] [jar:jar {execution: default-jar}]
[INFO] Building jar: /usr/local/google/bmanes/clients/lru_jdk5/build/
concurrentlinkedhashmap-lru-1.0_jdk5.jar
[INFO] [install:install {execution: default-install}]
[INFO] Installing /usr/local/google/bmanes/clients/lru_jdk5/build/
concurrentlinkedhashmap-lru-1.0_jdk5.jar to /home/bmanes/.m2/repository/com/
googlecode/concurrentlinkedhashmap/concurrentlinkedhashmap-lru/1.0_jdk5/
concurrentlinkedhashmap-lru-1.0_jdk5.jar
[INFO] [deploy:deploy {execution: default-deploy}]
Uploading: java-net:/maven2-repository/trunk/repository//trunk/repository/com/
googlecode/concurrentlinkedhashmap/concurrentlinkedhashmap-lru/1.0_jdk5/
concurrentlinkedhashmap-lru-1.0_jdk5.jar
[INFO] Uploading project information for concurrentlinkedhashmap-lru 1.0_jdk5
[INFO] Retrieving previous metadata from java.net-maven2-repository
[INFO] Uploading repository metadata for: 'artifact
com.googlecode.concurrentlinkedhashmap:concurrentlinkedhashmap-lru'
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1 minute 57 seconds
[INFO] Finished at: Fri Apr 30 21:07:33 PDT 2010
[INFO] Final Memory: 26M/207M
[INFO] ------------------------------------------------------------------------
I then browsed: http://download.java.net/maven/2/
Original comment by Ben.Manes@gmail.com
on 1 May 2010 at 4:41
How does the <dependencies> part of the POM work? Since the API jar does not
have
any external dependencies outside of the JRE, will Maven still pull down the
jars
needed for running the unit tests and benchmarks?
Original comment by zells...@gmail.com
on 1 May 2010 at 6:39
I think that it wouldn't, since those are scoped as testing. I excluded the
benchmark since there is no
mavenized version and will eventually replace it with Caliper. I couldn't test
since the artifact hasn't
materialized on the repository yet and my Maven skills are fairly poor. I might
need your help tuning
the pom.
Original comment by Ben.Manes@gmail.com
on 1 May 2010 at 6:49
Dang! Not everything is scoped as test, it looks like I'll have to upload a net
version Monday....
Original comment by Ben.Manes@gmail.com
on 1 May 2010 at 6:51
The artifact appeared in the repository. I will fix the POM on Monday to
properly
scope the test dependencies. Marking as 'done' now so people can start to play
with it.
Please let me know if you have any issues.
Original comment by Ben.Manes@gmail.com
on 1 May 2010 at 10:36
CLHM is now using a private Maven repository, instead of using a public one.
There's an annoying issue with this: it prevents projects that use CLHM as a
Maven dependency from being deployed in a public repository. Public
repositories generally require project dependencies to be available in a public
repository.
I'm currently using the source of CLHM in my project, which I'd really like to
replace with a Maven dependency. Is there any way that this project can moved
into a public repository?
Original comment by Guus.der...@gmail.com
on 28 Jul 2010 at 5:39
Guus, you can use the Java.net public repository
(maven2-repository.dev.java.net).
In issue #16 it was requested to switch away from java.net to the current one.
I can ensure both are maintained. I'd like to keep everyone happy.
Original comment by Ben.Manes@gmail.com
on 28 Jul 2010 at 5:45
Ben, I appreciate the efforts you're putting into this (I myself find these
kind of issues annoying at best), but I'm afraid that the java.net repo doesn't
help us out for this particular problem.
The Java.net repository does not sync with the Central Maven repositories. In
other words: if I would like to add CLHM as a dependency to my project, I still
need to add a <repository> element in my project pom (which prevents me from
uploading my project into any of the repos that sync with the central
repository, as my project is no longer 'self-contained', as I described
earlier).
See http://maven.apache.org/guides/mini/guide-central-repository-upload.html
for (a bit) more details.
Original comment by Guus.der...@gmail.com
on 28 Jul 2010 at 7:28
Okay, thanks for the clarification. My maven kung fu isn't strong.
I'll try to get this resolved by Friday evening.
Original comment by Ben.Manes@gmail.com
on 28 Jul 2010 at 5:21
Sorry for not getting to this on Friday or over the weekend. I was too
exhausted from work.
I kicked off the process and it will take up to 2 working days for them to host
the project. The ticket is:
http://issues.sonatype.org/browse/OSSRH-679
Original comment by Ben.Manes@gmail.com
on 3 Aug 2010 at 1:00
Did you manage to publish your artifacts but didn't they pop up in the
repository yet (which is happening to me for a unrelated project), or didn't
you get around to do this yet? If so: can I help out in any way?
Original comment by Guus.der...@gmail.com
on 8 Aug 2010 at 1:39
Sorry for the delay - bad timing as I've been busy.
After fighting with it, I finally got it to deploy. However its forcing it to
be marked as v1.1, despite repeatedly telling it to be v1.0.
https://oss.sonatype.org/content/groups/public/com/googlecode/concurrentlinkedha
shmap/concurrentlinkedhashmap-lru/1.1-SNAPSHOT/
I'll fight with it some more tonight, but this sure does remind me why I
dislike maven. :)
Original comment by Ben.Manes@gmail.com
on 10 Aug 2010 at 6:00
Okay, after a lot of pain both v1.0 and v1.0_jdk5 are on sonatype. Please
verify.
Original comment by Ben.Manes@gmail.com
on 10 Aug 2010 at 7:37
Hi Ben - Thanks for all of your hard work - I've been battling Maven releases
myself in the past few weeks, I can relate to how much of a pain this can be.
Provided that you have done the 'release' bit, you'll need to update the
OSSRH-679 issue with a request to activate the central sync. This will trigger
a one-time human check (which is required). If everything is OK, the guys at
Sonatype will activate an automatic sync, which you can then use until the end
of times. More detals are on
https://docs.sonatype.org/display/repository/sonatype+oss+maven+repository+usage
+guide (refer to step 11).
Original comment by Guus.der...@gmail.com
on 10 Aug 2010 at 7:47
Hi Guus,
According to the Sonatype guys the sync'ing has been enabled. It should be
pushed by now, but I haven't checked yet. Can you verify?
Original comment by Ben.Manes@gmail.com
on 11 Aug 2010 at 10:43
Things work rather brilliantly now, both on my local computer as in our
continuous integration system! Thanks again for all of the effort you've been
putting into this!
Original comment by Guus.der...@gmail.com
on 13 Aug 2010 at 7:41
Awesome, thanks for prodding me. :)
Does this mean that I can remove the repository from the front-page or does the
URL need to be updated?
Original comment by bma...@google.com
on 13 Aug 2010 at 7:47
Anyone can use the artifacts of this project now by simply defining it as a
dependency in their projects POM, like this:
<dependency>
<groupId>com.googlecode.concurrentlinkedhashmap</groupId>
<artifactId>concurrentlinkedhashmap-lru</artifactId>
<version>1.0_jdk5</version>
</dependency>
It might be helpful for people if you leave a message to this extend on your
project page, but all of the additional references to other repositories are no
longer needed.
Original comment by Guus.der...@gmail.com
on 13 Aug 2010 at 8:04
This is awesome. thanks Ben for making this happen
Original comment by ryan.daum
on 13 Aug 2010 at 11:30
Original issue reported on code.google.com by
lne...@gmail.com
on 17 Feb 2009 at 12:54