junit-dep not uploaded to Maven repository since 4.5

[MichaelHackett]

I see versions 4.6 and 4.7 of the bundled-libraries version of JUnit in the Maven Central repo, but no junit-dep. Can this be added by whoever is doing the uploads?

[bloritsch]

What's in JUnit dep?

[MichaelHackett]

Somewhat ironically, junit-dep is JUnit without its dependencies, in this case, Hamcrest. Whereas the standard JAR bundles a particular version of Hamcrest in with the JUnit classes, the -dep version is just the JUnit classes, with the Hamcrest dependency specified in the POM. (Maybe the "dep" refers to making the dependencies explicit.)

We use the junit-dep version because we want to use Hamcrest 1.2 with JUnit and Mockito, which I believe is only possible if Hamcrest is separated from the JUnit library.

Actually, now that I think about it, it might be best to simply change the junit library to this format and ditch the junit-dep version. This would be the preferred way of producing a library for the Maven repository -- to specify your dependencies in the POM, rather than bundling them into your JAR. Maven makes pulling those dependencies automatic, so bundling everything into one JAR isn't a convenience and may actually be a hindrance. Doing this should not break any existing uses because Maven will download and include the default version of Hamcrest automatically, unless it is already being overridden (as in our case).

Now I'm thinking I should perhaps change my request. I think the above might be less work than maintaining and uploading two Maven versions, although the Maven version will still be different than the standard download (which presumably will still bundle in Hamcrest).

[lukewpatterson]

related info: https://issues.sonatype.org/browse/OSSRH-166?focusedCommentId=110853&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_110853

[atagunov-xx]

JMock still depends on JUnit 4.4 - because it wants junit-dep That's a shame - pls either remove hamcrest classes from main junit jar or resume uploading latest junit-dep jars to maven central!

[akollegger]

+1

[dsaff]

I'm not a maven user, but willing to help make this happen. Let's start at the end. How will I know when this is working? Is there a command line I can run, or web page I can check? Thanks.

[dsaff]

Even better, can someone create a pull request with a shell script that I can run on release, so this just naturally happens?

[cgruber]

David, I can help out if you want. I have to get guava-gwt in order first, but I'm happy to get this working.

[dsaff]

Hey, cgruber, good to see you out here. twitter/@twoqubed has also volunteered to help. However, anyone who thinks they can get something done in 2 working days or less is welcome to grab the lock. Thanks!

[twoqubed]

As of now, how are artifacts being pushed to the main Maven repo? Are the artifacts built and signed as part of a build process and then a manual pull request issued? If so, it sounds like automating this as part of the release process would be good.

Also, it sounds like there is some confusion re: junit.jar and junit-deps.jar. To me, the names are indeed misleading. Not sure if this is something worth changing/fixing? I seems like the JAR with dependencies could just go away from a Maven perspective. Shouldn't having the POM dependency on hamcrest be enough? That way, users can override JUnit's hamcrest version by explicitly specifying another version in their POM?

I guess my main question is...what are we trying to accomplish :-)

[MichaelHackett]

As I wrote in my old comment, I think it would be less confusing to get rid of the version that bundles the dependencies and rely on the POM to pull them in.

However, is there a backward-compatibility issue here? If junit-deps.jar becomes junit.jar, will this break anyone's build? Ideally, developers are putting specific version numbers in their POMs, in which case there shouldn't be a problem, except to let everyone know about the change when they upgrade. But if they haven't pegged their junit to a particular version, and if, say, you've already specified a different version of Hamcrest in your own POM, how will Maven resolve any conflict in version numbers?

[dsaff]

@twoqubed,

Right now, artifacts are being pushed to the Maven repo when mysterious benefactors (I believe David Gageot, most recently), deign to bless me by doing so. I don't know how they are doing it. I assume that they are taking the junit-X.X.jar and junit-dep-X.X.jar and uploading them manually.

The (likely misguided) justification behind junit-dep was as an anti-analogue to projects like cglib, where cglib.jar did not include the dependencies, and cglib-nodep did. In truth, I wish we'd never depended on, nor included, hamcrest in the first place, and I'm likely to soon propose deprecating the dependency, and then killing it, in favor of an assertThat implementation in junit.contrib, which will be a well-behaved maven project.

Until then, my feeling is to tackle this in two or three steps.

Step one: make sure that the projects that are already being pushed to maven are kept up to date, with their current manifests (junit = junit.* + org.junit.* + org.hamcrest., junit-dep = junit - org.hamcrest.). This seems a monotonic win: the projects are already out there, outdated, and it can only be better to bring them up to date. Step two: Potentially rename the projects to ease confusion (This is where @MichaelHackett's concerns come in) Step three: Potentially delete the omnibus (junit + hamcrest) project altogether.

Make sense? Thanks,

David

[dsaff]

Sorry, meant:

Thanks a lot! (I really appreciate the thought going into this).

[MichaelHackett]

Makes sense to me, David.

And it just occurred to me a way to avoid compatibility problems when renaming junit-sans-hamcrest. Just as the Java code (IIRC) has been migrating from using junit.xxx to org.junit.xxx for packages, perhaps the new junit-only Maven package could be placed in the org.junit group, and EOL the old name. Then a developer would have to change their project POM to get the new version, so nothing would break from an automatic update.

Of course, if you simply remove the Hamcrest dependency altogether, that also solves the problem. :-)

[twoqubed]

Sounds good. Then my plan is to...

1. Get my head around what is currently happening in the Maven build. From an initial glance it is simply the minimum - producing a default JAR.
2. Update the build so that the desired artifacts are produced. This would be both jar files (junit and junit-dep), plus all of the accompanying artifacts (source, javadocs, etc).
3. Figure out just how much of the process of uploading a 3rd-party artifact to the central Maven repository can be automated. The process is documented here:

https://docs.sonatype.org/display/Repository/Uploading+3rd-party+Artifacts+to+Maven+Central

[cgruber]

Ok. twoqubed, I've done the uploading on behalf of google for guava and the like. If you get the thing so that it builds and signs (maybe david can create a signing pgp cert) then I can walk you through what we do.

The short version is that you set up a nexus (maven repo management software) account on oss.sonatype.org, and get the sonatype folks to give you permissions to push to that groupId. Then, you set the deployment descriptor to push to a virtual repo location they give you that nexus turns into a sort of candidate repository. You then go in, promote the candidate to release, and it syncs to the main repo.

If you like, I am happy to have you set up creating the right artifacts such that you could safely "mvn install" them, and I can do the other part (signing and repository config). Ultimately, there's a manual step in the maven repository management software where you MUST promote the temporary repo image so it can be pushed to hte public, but it's a login, look it over, close it, promote it. It's about 10 minutes work.

[cgruber]

Probably, we can end up getting the signing into the continuous integration system, too, so it's nearly all automagical.

[twoqubed]

I have been tinkering with the Maven build trying to produce the two primary artifacts:

1. junit-4.x.jar - The jar with the hamcrest dependencies
2. junit-4.x-dep.jar - The jar without the hamcrest dependencies

As it turns out, this is really fighting against Maven's conventions. In fact, building a jar with the default name of the project that contains additional classes while building a jar with a qualifier that only contains the project's classes is the opposite of Maven's conventions.

So, it seems like there a few options...

First, swim against the grain and coerce Maven to continue producing the same artifacts that have been produced in the past.

Second, have Maven build the base artifact(s) and use scripts to produce the additional artifacts. This could potentially be more straight forward than trying the bend Maven over backwards.

Third, start building just the stand alone jar starting at 4.9 (i.e. junit-49.jar that would just contain junit classes). This is clearly the most straight forward route. The question is what are the repercussions of this approach? Well, there are two likely scenarios...

A user of of junit-4.x upgrades to junit-4.9. If the hamcrest classes were in their classpath before simply through the junit jar, they should continue to be in their classpass transitively through junit's dependency.

A user of junit-4.x-dep attempts to upgrade to junit-4.9-dep, but is forced to use junit-4.9 instead. Once this transition is done, their upgrade should be seamless. In fact, a POM file for junit-4.9-dep could be produced that clarifies the change for users.

I certainly prefer the third option, but I would like to hear what others think.

[cgruber]

I'd +1 option #3. It seems the cleanest option.

[kueblboe]

I strongly favor option 3 (adhering to Maven standards) as well. One issue with this approach is though that you might piss off non-Maven users, who will have a missing dependency once upgrading to junit-4.9. But those folks should be used to dependency issues anyway... :)

[twoqubed]

I have been working with the (perhaps faulty) assumption that the main users of this change would be Maven users. I assumed non-Maven users would likely be getting the JUnit jar my other means, like downloading it directly from the JUnit distribution page.

I either case, I still think it makes sense to move towards a less confusing packaging and distribution.

[kueblboe]

I did see that there is also a build.xml. But I question why there are several ways to produce the junit.jar and would certainly prefer, if there was only one. Otherwise you end up with different artifacts that have the same name. If the one and only way was indeed Maven, then that would mean that all junit users are influenced by your changes. That would also mean to get rid of the build.xml.

Even when I (have to) work on a project that is using Ant, I usually get required libraries from Maven Central. Just because it's easier to search.

As I said before, I agree on going for the standard Maven way. After all, these standards (amongst other things) are what make Maven so attractive. But, I guess, in the end you really need David's and Kent's agreement more than mine.

[MichaelHackett]

I thought David's suggestion of a staged approach was a wise one, but if it's too difficult to automate a build of the existing packages, then I would reiterate my suggestion of changing the group ID (to be more in line with standard practice as well) if you are also changing what is in the "junit" package (to be just junit classes, without its dependencies included). Then no one's builds can break until they explicitly change the JUnit group ID in their project's POM.

[cgruber]

It's worth noting that we can also re-work the build.xml to use maven for dependency grabbing, but still allow people to use ant to build if they prefer. Though there's limited value in this given that most people download binaries.

[twoqubed]

@MichaelHackett - I like that approach as well. The contents of the 4.9 POMs for the legacy groupId's artifacts good point to the new groupId as well. This would at least give users upgrading to 4.9 some breadcrumbs as the where to find the latest release.

[nealeu]

+1 on #3 here

If hamcrest is made a dependency in 4.9, then when I change my junit artifact from 4.8.2 to 4.9, then I won't notice. Those who use junit-dep, would, as there would not be a 4.9 version of junit-dep (not great loss as it's been missing a few years from the maven perspective).

From this point forwards, anyone using JUnit 4.9 would be using just the single artifact.

[dsaff]

Woah. Look away from e-mail for the weekend, and look how far behind you can get...

I'd strongly prefer a solution that takes jars built from ant, and uploads them to the maven repository. It looks like some of the above solutions have suggested switching over the jar-building tasks to ant. There might be advantages to that, but I'd like to consider them outside of this issue.

I'd also strongly prefer that we leave the jars alone for people who tend to grab them from anywhere but through maven. If I downloaded junit-4.8.2.jar from github, then I have reason to expect that junit-4.9.jar, coming from the same place, would have the same contents.

Is there a way to indicate "deprecated" groupIds in maven? Could we produce 4.9 junit and 4.9 junit-dep with the same contents, but have them both deprecated, with a note to use the newly-created, cleanly-defined org.junit groupId for maximum future happiness?

[twoqubed]

@dsaff Let me see if I understand what you are saying...

Ant is the build tool of record for JUnit, so ideally all artifacts, including those meant to be published to a Maven repository, should be generated from the Ant build. If this is the case, that should not be that difficult.

And it sounds like the initial goal of this thread was to extend that process to produce not only the Maven artifacts, but a bundle suitable to being pushed to the central Maven repository, helping automate the release process. Is that correct?

So...going forward, let's assume that the artifacts (contents and names) will remain the same for non-Maven artifacts, such as those published directly on github. For Maven, we can take the artifact created by Ant that does not contain the hamcrest dependencies and begin publishing that to a new coordinate org.junit:junit:4.9.

We can also publish a POM to the legacy coordinate junit:junit:4.9 that indicates its deprecated nature. Maven supports this via:

http://maven.apache.org/pom.html#Relocation

If we want, we could still publish a JAR file to Maven with hamcrest bundled with it, but give it a "classifier" (Maven term). But it sounds like you would prefer to move away from this, especially since Maven's transitive dependencies will take care of this anyway.

[MichaelHackett]

@twoqubed, re: Relocation: Nice find! I didn't even know that existed. I think that would be ideal.

Question, though. You wrote (emphasis mine):

And it sounds like the initial goal of this thread was to extend that process to produce not only the Maven artifacts ...

It seems like you meant to refer to the non-Maven builds here (i.e. the JARs that are published on the JUnit.org website for direct download). Or am I misunderstanding your intention?

[twoqubed]

@MichaelHackett

I guess I assumed that the process for building non-Maven artifacts is working just fine. What needs improving/automating is the production of Maven artifacts. From what I have inferred from this discussion, this has been a somewhat manual process taken on by good Maven citizens :-)

So, not only does the building of Maven artifacts need to be automated, a good additional step would be to bundle all of the Maven artifacts (presumably produced by the Ant build) into a file suitable for pushing to the Maven central repo.

So, I did mean "Maven" artifacts, but I clearly did not express my thinking well :-)

My goal is to dig into the Ant build this evening to see what is produced there. It all necessary artifacts are being build by Ant (javadoc, source and binary JARs), then these can all easily be transformed into their Maven counterparts.

Also, it looks like the pom.xml may exist simply to be pushed to the Maven repository, and not as an actual build file. In other words, all builds are done with Ant and the pom.xml is there to be included for Maven users. (@dsaff, please correct me if I am off base here). If that is indeed the case, perhaps renaming the file could reduce the chance of any confusion that JUnit is build with Maven. Perhaps pom-template.xml or some such?

[dsaff]

@twoqubed:

You said: "Ant is the build tool of record for JUnit, so ideally all artifacts, including those meant to be published to a Maven repository, should be generated from the Ant build." Yes, that's what I meant.

Actually, I believe the OP really just wanted someone to upload a new junit-dep jar to maven. Relying on the kindness of strangers for our maven uploads has worked fairly well, but not perfectly, so far, so I was hoping someone could check in a script to the codebase that I could run when publishing a release, to just automatically (or as automatically as possible) bring maven up to date.

You said: "all builds are done with Ant and the pom.xml is there to be included for Maven users." Yes, that's the case. And I'm happy to rename the file if that makes things less confusing for maven users.

Thank you for pushing on this. If you come up with a pull request that makes sense, let's float it on junit@yahoogroups.com to get some wider feedback.

Also, does anyone know if maven or surefire has a hard-coded dependency on a particular junit groupId anywhere, such that changing it would break maven itself? Any ideas whom to ask?

[twoqubed]

I made some pretty good progress tonight on creating a script to automate the generation of the Maven artifacts. I've pushed it to my JUnit fork here:

https://github.com/twoqubed/junit

A few notes...

1. The one thing I was not sure how to get around was the need to have a tool (GPG in this case) installed locally in order to sign the individual artifacts. The script makes the assumption that the user has gpg in their path. There may be a way to reduce the dependency, but I thought this was a reasonable first pass.
2. As of now, the Maven artifacts generated follow the exact same convention as exists today. Two artifacts are created, junit (with hamcrest) and junit-dep (without hamcrest). After @dsaff's comments, it sounded like we should take the "one step at a time" approach.
3. The script is also completely integrated into Ant. You should be able to execute:

ant clean dist prepare.maven

And have all required artifacts produced.

I would love to have some others take a peek and make any suggestions or improvements.

[dsaff]

Ryan (@twoqubed),

Poking around, it looks great. Can you open a pull request and link it here, to make it easier to have a thread focused line-by-line on the code?

[twoqubed]

@dsaff

Done. I am still new to git and github, so please let me know if I didn't get this right.

https://github.com/KentBeck/junit/pull/203

[dsaff]

So, coming back to this, it looks like I need permissions to upload: https://issues.sonatype.org/browse/OSSRH-1493. When that's resolved, we'll try to move forward.

[cgruber]

You and twoqubed seem to have this covered, David, but if you need anything, let me know.

On Mar 24, 2011, at 11:37 AM, dsaff wrote:

So, coming back to this, it looks like I need permissions to upload: https://issues.sonatype.org/browse/OSSRH-1493. When that's resolved, we'll try to move forward.

Reply to this email directly or view it on GitHub: https://github.com/KentBeck/junit/issues/33#comment_912734

[dsaff]

@twoqubed, after review, I think that the end-to-end process would be easier if it were in ant. I'm going to take a stab at translating your script into ant. Everything would be easier if our build so far wasn't already in ant...

[dsaff]

First commit in this direction: https://github.com/KentBeck/junit/commit/d148e1a26378d54b8fdff4afd5d5c4c35ad57885 Puts a test around current jar-building behavior

[dsaff]

After the commit here:

https://github.com/KentBeck/junit/pull/213

I appear to be able to automatically upload staged repositories. However, I don't seem to be able to promote them through the web interface:

https://issues.sonatype.org/browse/OSSRH-1545

[dsaff]

OK, it turns out I misread the sonatype docs. 24 hours to comment on the pull:

https://github.com/KentBeck/junit/pull/213

And then I push it and close this, finally.

[dsaff]

Pushed and closed.