search

easyfmxu / concurrentlinkedhashmap

Automatically exported from code.google.com/p/concurrentlinkedhashmap
Apache License 2.0
0 stars 0 forks source link

deploy concurrentlinkedhashmap to public Maven repository #3

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
Current Behavior:
Download and "install" jar and manually.

Desired Behavior:
Add Maven depdendency to pom.xml and begin using this excellent
implementation.  This will make this library (and any of its updates)
trivially accessible which will very likely increase its use.

Complexity / Amount of Change Required:
Some complexity, but minimal code (pom.xml).  The more willingness to adopt
the "Maven way", the easier this is.

Original issue reported on code.google.com by lne...@gmail.com on 17 Feb 2009 at 12:54

GoogleCodeExporter commented 9 years ago

Original comment by Ben.Manes@gmail.com on 18 Feb 2009 at 8:22

GoogleCodeExporter commented 9 years ago 
I hope to do this once I'm no longer releasing a public snapshot. This may be 
fairly 
soon, as I finally got over the last algorithmic design hurdle. Right now its 
more 
of a "have fun, but use at your own risk" type of deal.

Original comment by Ben.Manes@gmail.com on 27 Feb 2009 at 6:50

GoogleCodeExporter commented 9 years ago

Original comment by Ben.Manes@gmail.com on 28 Feb 2009 at 9:03

GoogleCodeExporter commented 9 years ago

Original comment by Ben.Manes@gmail.com on 25 Mar 2009 at 2:49

GoogleCodeExporter commented 9 years ago 
Hey, cool work on this project...

I desperately need something like your implementation for jmemcached (see 
http://code.google.com/p/jmemcache-daemon). So question is...

1. How close are you to a public release?
2. How close are you to having a release in a public Maven repository?

Original comment by ryan.daum on 21 Aug 2009 at 3:14

GoogleCodeExporter commented 9 years ago 
I just haven't had any time to work on it for the last few months. When I left 
off, 
my unit tests showed a race condition that I was having a hard time debug'ing. 
It 
was a bit nasty in that the condition was repeatable but challenging to 
determine 
the order of events creating it. I just haven't had any time to get back to it, 
though. I could instead supply you with a tweaked version that uses a 
ReentrantLock 
around the link chain to ensure correctness, but have a small critical section 
vs. 
synchronizing a LHM.

In your case, since you have one very large cache, you either want a 
soft-reference 
based approach (e.g. Google Collections) to rely on GC's LRU strategy, or a 
striped 
LRU facade. You need LRU due to the generational approach often used with 
memcached 
to simulate tagged flushing, so a SecondChanceFifo would probably suffer by not 
being able to tell which is really a hot entry. Simply create an array of LRU 
maps 
and hash to one by the key. With a good spreading function (see CHM) you'll get 
close to a true LRU but split the contention across multiple locks. I can 
quickly 
whip that up for you, if you'd like.

Original comment by Ben.Manes@gmail.com on 21 Aug 2009 at 4:29

GoogleCodeExporter commented 9 years ago 
Thanks for your response.

The map I need needs to provide the following characteristics:

   * Likely some type of 'striped' locking as you describe in your wiki page, as much as 
I understand it. Basically, to improve concurrency, it should not be necessary 
to lock 
the entire store for  operations that affect one entry only
   * The ability to perform several operations with one lock (i.e. for CAS or INCR/DECR, 
a get followed by read, with a write lock for the whole operation).
   * It ought to be possible to expire items not only in terms of a LRU policy, but also 
when the size of the items exceeds the maximum declared cache size.

So there are a number of constraints, and I fear I'm not qualified to write the 
data 
structure myself. What's there now is far from efficient under heavy concurrent 
threading because it relies on a global lock. 

Is the last point even possible with your class & its ExpirationPolicy?

Original comment by ryan.daum on 21 Aug 2009 at 6:08

GoogleCodeExporter commented 9 years ago 
[deleted comment]
GoogleCodeExporter commented 9 years ago 
A CAS is supported, and a increment/decrement with a get followed by an CAS 
increment (e.g. AtomicLong). I doubt that it needs to be locked throughout 
those 
operations.

The size is something that I've wanted to add for fun, but its a non-core 
feature. 
Basically I would add the ability to supply a "weight" for an entry and be used 
when 
determining whether the capacity was exceeded. This is useful for caching 
collections, which risk being unbounded lists by weight an element by its 
list's 
length. For memory, you would need to enable agent instrumentation (e.g. see 
SizeOf 
project), which will be a faster than serializing to a byte[] to get its length.

Memcached also supports time-based eviction (expiration) which you'll need to 
handle. I have that through a decorator, which I should probably post on the 
wiki...

If you want help we can continue the discussion in the group or email to not 
divert 
the discussion of this task.

Original comment by Ben.Manes@gmail.com on 21 Aug 2009 at 7:44

GoogleCodeExporter commented 9 years ago 
O.k. I'll contact you re: this topic off-issue.

But on topic for this issue, if you want me to set up a Maven build for you, I 
can do 
that...

Original comment by ryan.daum on 21 Aug 2009 at 10:19

GoogleCodeExporter commented 9 years ago 
Oh sure, that would be really great. :)

Original comment by Ben.Manes@gmail.com on 21 Aug 2009 at 10:26

GoogleCodeExporter commented 9 years ago 
From: Ian Clarke <ian.cla...@gmail.com>
To: concurrentlinkedhashmap@googlegroups.com
Subject: Re: [CLHM] Re: Maven repository

Ben,

Might I suggest this: https://maven2-repository.dev.java.net/

This is what Google recommend that Google Code projects use.

Kind regards,
Ian.

Original comment by Ben.Manes@gmail.com on 10 Nov 2009 at 8:54

GoogleCodeExporter commented 9 years ago 
Sonatype offers hosting & mirroring to maven central for open source projects 
and they 
have step-by-step instructions on how to do it:

http://nexus.sonatype.org/oss-repository-hosting.html

Original comment by ryan.daum on 27 Nov 2009 at 3:27

GoogleCodeExporter commented 9 years ago 
Thanks for the nudge! I've been swamped with starting a new job (Google), 
moving, 
etc. so I've been pretty slack on this. Now that its starting to settle down, 
I'll 
try to do this Tues/Weds evening.

Original comment by Ben.Manes@gmail.com on 1 Dec 2009 at 2:15

GoogleCodeExporter commented 9 years ago 
Hey Adam, can you take this one? We can publish clhm-release-1.0-lru.jar to a 
maven 
site and continue on with LIRS in trunk.

Thanks!
Ben

Original comment by Ben.Manes@gmail.com on 13 Apr 2010 at 5:51

GoogleCodeExporter commented 9 years ago 
Does this library run on JDK 1.5? or JRE 1.6 is mandatory? I have requirement 
to use
it on JRE 1.5, but when I compile on it it shows UnsupportedClassVersionError: 
Bad
version number in .class file.

Original comment by anandw...@gmail.com on 14 Apr 2010 at 10:08

GoogleCodeExporter commented 9 years ago 
There are no Java-6 features required. The advantages was that JDK5 is 
end-of-life, 
AbstractMap$SimpleEntry is public so it doesn't need to be duplicated, and 
AbstractCollection#toArray() 
was fixed to be threadsafe so it doesn't need to be overridden (keySet, values, 
entrySet).

A backport could be provided. Please open a new issue for that task.

Original comment by Ben.Manes@gmail.com on 14 Apr 2010 at 10:17

GoogleCodeExporter commented 9 years ago 
I followed the java.net guide and published jdk5/jdk6 versions of the v1.0-LRU 
release. While the "maven deploy" succeeded, I do not see anything when 
browsing the 
repository. I'll check again Monday in hopes that there is a staging process 
and, if 
not, will need someone with Maven skills to step in.

Maven users should be able to either of the following:

<!-- JDK 6 -->
<dependency>
  <groupId>com.googlecode.concurrentlinkedhashmap</groupId>
  <artifactId>concurrentlinkedhashmap-lru</artifactId>
  <version>1.0</version>
</dependency>

<!-- JDK 5 -->
<dependency>
  <groupId>com.googlecode.concurrentlinkedhashmap</groupId>
  <artifactId>concurrentlinkedhashmap-lru</artifactId>
  <version>1.0_jdk5</version>
</dependency>

I followed the guide and got the following output:
$ mvn deploy
...
[INFO] [jar:jar {execution: default-jar}]
[INFO] Building jar: /usr/local/google/bmanes/clients/lru_jdk5/build/
concurrentlinkedhashmap-lru-1.0_jdk5.jar
[INFO] [install:install {execution: default-install}]
[INFO] Installing /usr/local/google/bmanes/clients/lru_jdk5/build/
concurrentlinkedhashmap-lru-1.0_jdk5.jar to /home/bmanes/.m2/repository/com/
googlecode/concurrentlinkedhashmap/concurrentlinkedhashmap-lru/1.0_jdk5/
concurrentlinkedhashmap-lru-1.0_jdk5.jar
[INFO] [deploy:deploy {execution: default-deploy}]
Uploading: java-net:/maven2-repository/trunk/repository//trunk/repository/com/
googlecode/concurrentlinkedhashmap/concurrentlinkedhashmap-lru/1.0_jdk5/
concurrentlinkedhashmap-lru-1.0_jdk5.jar

[INFO] Uploading project information for concurrentlinkedhashmap-lru 1.0_jdk5
[INFO] Retrieving previous metadata from java.net-maven2-repository
[INFO] Uploading repository metadata for: 'artifact 
com.googlecode.concurrentlinkedhashmap:concurrentlinkedhashmap-lru'
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1 minute 57 seconds
[INFO] Finished at: Fri Apr 30 21:07:33 PDT 2010
[INFO] Final Memory: 26M/207M
[INFO] ------------------------------------------------------------------------

I then browsed: http://download.java.net/maven/2/

Original comment by Ben.Manes@gmail.com on 1 May 2010 at 4:41

GoogleCodeExporter commented 9 years ago 
How does the <dependencies> part of the POM work?  Since the API jar does not 
have
any external dependencies outside of the JRE, will Maven still pull down the 
jars
needed for running the unit tests and benchmarks?

Original comment by zells...@gmail.com on 1 May 2010 at 6:39

GoogleCodeExporter commented 9 years ago 
I think that it wouldn't, since those are scoped as testing.  I excluded the 
benchmark since there is no 
mavenized version and will eventually replace it with Caliper. I couldn't test 
since the artifact hasn't 
materialized on the repository yet and my Maven skills are fairly poor. I might 
need your help tuning 
the pom.

Original comment by Ben.Manes@gmail.com on 1 May 2010 at 6:49

GoogleCodeExporter commented 9 years ago 
Dang! Not everything is scoped as test, it looks like I'll have to upload a net 
version Monday....

Original comment by Ben.Manes@gmail.com on 1 May 2010 at 6:51

GoogleCodeExporter commented 9 years ago 
The artifact appeared in the repository. I will fix the POM on Monday to 
properly 
scope the test dependencies. Marking as 'done' now so people can start to play 
with it.

Please let me know if you have any issues.

Original comment by Ben.Manes@gmail.com on 1 May 2010 at 10:36

GoogleCodeExporter commented 9 years ago 
CLHM is now using a private Maven repository, instead of using a public one. 
There's an annoying issue with this: it prevents projects that use CLHM as a 
Maven dependency from being deployed in a public repository. Public 
repositories generally require project dependencies to be available in a public 
repository. 

I'm currently using the source of CLHM in my project, which I'd really like to 
replace with a Maven dependency. Is there any way that this project can moved 
into a public repository?

Original comment by Guus.der...@gmail.com on 28 Jul 2010 at 5:39

GoogleCodeExporter commented 9 years ago 
Guus, you can use the Java.net public repository 
(maven2-repository.dev.java.net).

In issue #16 it was requested to switch away from java.net to the current one. 
I can ensure both are maintained. I'd like to keep everyone happy.

Original comment by Ben.Manes@gmail.com on 28 Jul 2010 at 5:45

GoogleCodeExporter commented 9 years ago 
Ben, I appreciate the efforts you're putting into this (I myself find these 
kind of issues annoying at best), but I'm afraid that the java.net repo doesn't 
help us out for this particular problem. 

The Java.net repository does not sync with the Central Maven repositories. In 
other words: if I would like to add CLHM as a dependency to my project, I still 
need to add a <repository> element in my project pom (which prevents me from 
uploading my project into any of the repos that sync with the central 
repository, as my project is no longer 'self-contained', as I described 
earlier).

See http://maven.apache.org/guides/mini/guide-central-repository-upload.html 
for (a bit) more details.

Original comment by Guus.der...@gmail.com on 28 Jul 2010 at 7:28

GoogleCodeExporter commented 9 years ago 
Okay, thanks for the clarification. My maven kung fu isn't strong.

I'll try to get this resolved by Friday evening.

Original comment by Ben.Manes@gmail.com on 28 Jul 2010 at 5:21

GoogleCodeExporter commented 9 years ago 
Sorry for not getting to this on Friday or over the weekend. I was too 
exhausted from work.

I kicked off the process and it will take up to 2 working days for them to host 
the project. The ticket is:
http://issues.sonatype.org/browse/OSSRH-679

Original comment by Ben.Manes@gmail.com on 3 Aug 2010 at 1:00

GoogleCodeExporter commented 9 years ago 
Did you manage to publish your artifacts but didn't they pop up in the 
repository yet (which is happening to me for a unrelated project), or didn't 
you get around to do this yet? If so: can I help out in any way?

Original comment by Guus.der...@gmail.com on 8 Aug 2010 at 1:39

GoogleCodeExporter commented 9 years ago 
Sorry for the delay - bad timing as I've been busy.

After fighting with it, I finally got it to deploy. However its forcing it to 
be marked as v1.1, despite repeatedly telling it to be v1.0.

https://oss.sonatype.org/content/groups/public/com/googlecode/concurrentlinkedha
shmap/concurrentlinkedhashmap-lru/1.1-SNAPSHOT/

I'll fight with it some more tonight, but this sure does remind me why I 
dislike maven. :)

Original comment by Ben.Manes@gmail.com on 10 Aug 2010 at 6:00

GoogleCodeExporter commented 9 years ago 
Okay, after a lot of pain both v1.0 and v1.0_jdk5 are on sonatype. Please 
verify.

Original comment by Ben.Manes@gmail.com on 10 Aug 2010 at 7:37

GoogleCodeExporter commented 9 years ago 
Hi Ben - Thanks for all of your hard work - I've been battling Maven releases 
myself in the past few weeks, I can relate to how much of a pain this can be.

Provided that you have done the 'release' bit, you'll need to update the 
OSSRH-679 issue with a request to activate the central sync. This will trigger 
a one-time human check (which is required). If everything is OK, the guys at 
Sonatype will activate an automatic sync, which you can then use until the end 
of times. More detals are on 
https://docs.sonatype.org/display/repository/sonatype+oss+maven+repository+usage
+guide (refer to step 11).

Original comment by Guus.der...@gmail.com on 10 Aug 2010 at 7:47

GoogleCodeExporter commented 9 years ago 
Hi Guus,

According to the Sonatype guys the sync'ing has been enabled. It should be 
pushed by now, but I haven't checked yet. Can you verify?

Original comment by Ben.Manes@gmail.com on 11 Aug 2010 at 10:43

GoogleCodeExporter commented 9 years ago 
Things work rather brilliantly now, both on my local computer as in our 
continuous integration system! Thanks again for all of the effort you've been 
putting into this!

Original comment by Guus.der...@gmail.com on 13 Aug 2010 at 7:41

GoogleCodeExporter commented 9 years ago 
Awesome, thanks for prodding me. :)

Does this mean that I can remove the repository from the front-page or does the 
URL need to be updated?

Original comment by bma...@google.com on 13 Aug 2010 at 7:47

GoogleCodeExporter commented 9 years ago 
Anyone can use the artifacts of this project now by simply defining it as a 
dependency in their projects POM, like this:

<dependency>
    <groupId>com.googlecode.concurrentlinkedhashmap</groupId>
    <artifactId>concurrentlinkedhashmap-lru</artifactId>
    <version>1.0_jdk5</version>
</dependency>

It might be helpful for people if you leave a message to this extend on your 
project page, but all of the additional references to other repositories are no 
longer needed.

Original comment by Guus.der...@gmail.com on 13 Aug 2010 at 8:04

GoogleCodeExporter commented 9 years ago 
This is awesome.  thanks Ben for making this happen

Original comment by ryan.daum on 13 Aug 2010 at 11:30