aQute/lib/collections/MultiMap name clash with HashMap, >= Java 8 issue

[wltjr]

Trying to build what is found in this jar, seems the sources only exist here I could not find them anywhere else. https://github.com/bndtools/bnd/tree/master/cnf/repo/aQute.jpm.clnt

I Ran into some import errors I corrected with specific imports vs wildcard "error: reference to Base64 is ambiguous both class aQute.lib.base64.Base64 in aQute.lib.base64 and class java.util.Base64 in java.util match"

Those were easy to fix but then ran into a Java 8 issue, OSGI-OPT/src/aQute/lib/collections/MultiMap.java:84: error: name clash: remove(K,V) in MultiMap and remove(Object,Object) in HashMap have the same erasure, yet neither overrides the other public boolean remove(K key, V value) { ^ where K,V are type-variables: K extends Object declared in class MultiMap V extends Object declared in class MultiMap

Which I see mentioned here http://stackoverflow.com/questions/23785807/java-name-clash-error-despite-different-method-signatures

I tried the suggested way of renaming, which allows it to compile but I do not think is the proper way to address this issue. MultiMap.remove(K key, V value) -> removeKeyValue(K key, V value)

Not sure if the remove method in HashMap can be used instead of the one in MultiMap. But this would likely break anything that needed the MultiMap.remove method, as it will end up using HashMap remove instead.

Another way to go is change it from remove(K key, V value) -> remove(Object key, Object value)

Since both K and V extend object, I think that might be valid. Though could be another proper way to address this issue. Most just use the pre-compiled classes in those jars so it is not an issue. I would like to address this the proper way vs some hack that allows it to compile but could break usage.

P.S. It would be nice if the code was not in jars only, so patches and/or pull requests could be provided. There seems to be allot of stuff used by bnd found only in jars. Likely something for another issue/ticket. Only place I could find sources was in the jars here https://github.com/bndtools/bnd/tree/master/cnf/repo https://github.com/pkriens/aQute.repo/tree/master/repo

[wltjr]

Here is the patch I made, showing all changes and I went with the remove(K key, V value) -> remove(Object key, Object value) for now, though not sure that is correct. https://github.com/Obsidian-StudiosInc/os-xtoo/blob/master/dev-java/aqute-jpm-clnt/files/aqute-jpm-clnt-2.2.0-fix-imports.patch

[bjhargrave]

Those were easy to fix but then ran into a Java 8 issue, OSGI-OPT/src/aQute/lib/collections/MultiMap.java:84: error: name clash: remove(K,V) in MultiMap and remove(Object,Object) in HashMap have the same erasure, yet neither overrides the other public boolean remove(K key, V value) {

I don't know what version of the source you are using, but you should probably try the latest in this repo. This problem has been fixed in bnd 3.0 by https://github.com/bndtools/bnd/commit/64d3af08777a3acc6a2f2184ebcb627fed8b7655.

aQute.jpm.clnt

The source for this is not part of this repo. It is a binary provided by @pkriens.

[wltjr]

I am using this with bnd 3.1.0. I am using the latest tag. https://github.com/Obsidian-StudiosInc/os-xtoo/blob/master/dev-java/bnd/bnd-3.1.0.ebuild

I think this should be re-opened as it is an issue. The sources for this ARE in this repo, they are just in binary format within the jar. There are several sources that can be found in binary jars within this repo and 1 other that BND depends on.

See this is in BND, in the very same repo I am getting the sources for BND https://github.com/bndtools/bnd/tree/master/cnf/repo/aQute.jpm.clnt

The latest BND 3.1.0 for Gentoo https://github.com/Obsidian-StudiosInc/os-xtoo/blob/master/dev-java/bnd/bnd-3.1.0.ebuild

This is the package to build aQute.jpm.clnt from the binary jar that resides within BND https://github.com/Obsidian-StudiosInc/os-xtoo/blob/master/dev-java/aqute-jpm-clnt/aqute-jpm-clnt-2.2.0.ebuild

You cannot build BND without this jar, and the jar is in the repo. The difference in my case is that I am compiling the sources in that jar. When BND is built using gradle, it uses the binary/jar and is not compiling all sources. I had to use gradle to build bnd to find out where the missing classes were, and when I did I found the sources in the jar.

Please re-open as this is an issue. Sources exist, they are in the BND repo, just in a binary jar, and anyone who compiles it with Java >- 1.8 will have the same issues. I checked the pre-built classes in that jar. They are presently built using JDK 1.5. At some point it might be recompiled with a newer JDK, and this issue will come up again.

[wltjr]

Looks like it was fixed in libg, but the sources for this exist else where. It seems allot of classes can be found in more than one jar, for not only aQute stuff, but the OSGi bundles as well. Splitting it out so things are not duplicated is not trivial.

While it was fixed in libg. BND even the lastest 3.1.0, also ships the unfixed version that is pre-compiled. Not sure if that will be phased out at some point. But it is presently used and needed by aQute-repository. This jar in the BND repo that gradle uses -> https://github.com/bndtools/bnd/blob/master/cnf/repo/aQute.jpm.clnt/aQute.jpm.clnt-2.2.0.jar

I had to compile aQute.jpm.clnt to get the following aQute.jpm.facade.repo,\ aQute.jsonrpc.proxy,\ aQute.rest.urlclient,\

Per the build dependencies here https://github.com/bndtools/bnd/blob/master/biz.aQute.repository/bnd.bnd

Thus while fixed in libg, it also exists else where. I guess I can try adding libg as a dependency and not compiling those aspects of the sources in aQute.jpm.clnt-2.2.0.jar

[bjhargrave]

You are attempting to rebuild the source code of bnd without using the build tools environment in which it was built. This is a massive challenge which is basically unnecessary since you can just get this complete repo and build it as it is designed. As for building from the source code embedded in jars, that is not really the purpose of the source code in the jars. That is for reference during debugging. Necessary information to build the jar from the source in the jar is missing. Such as the bnd files and any buildpath dependencies.

The aQute.libg project is built and the other bundles being built in this repo use bnd to copy necessary aQute.libg packages into the bundle. That is, a bundle's use of aQute.libg is in internal implementation dependency. This is why they will have their own copy rather than importing it.

aQute.jpm.clnt is a binary dependency for this repo. This repo does not need nor include the source code. You will need to ask Peter Kriens if you want the source build for aQute.jpm.clnt.

I am sure you can appreciate that this repo builds correctly. We have CI builds on Travis and Jenkins. So if there are problems in the code in the repo which we should fix to improve future releases of bnd, I would like to know about them so they can be addressed. But given what you have raised here, the source code for MultiMap, https://github.com/bndtools/bnd/blob/master/aQute.libg/src/aQute/lib/collections/MultiMap.java#L108, has already been fixed to address the issue since bnd 3.0. Since the problem in the source code has already been fixed, I am not sure what else there is to do for this issue.

[bjhargrave]

I am using this with bnd 3.1.0. I am using the latest tag. https://github.com/Obsidian-StudiosInc/os-xtoo/blob/master/dev-java/bnd/bnd-3.1.0.ebuild

bnd requires bnd to be built. It is "self-hosted". This is how the build for this repo works. I am not sure how you will build this in some other build environment.

[wltjr]

I know this was a big challenge. I have bnd, bndlib, libg, and a bunch of other pieces packaged in order to build bnd from source, including all dependencies. They can all be built now, without using the gradle build system. I haven't messed with building or running tests, so not sure about that. But since each piece depends on others. If one was not built, the others would not build.

Gentoo Linux is an operating system that builds all open source software from source. There are many reasons for such. Here is a document I made many years ago that covers most of the reasons. https://wiki.gentoo.org/wiki/Project:Java/Why_build_from_source

I understand there is a build system. I used it to help find the missing classes for packaging and building. The build system relies heavily on lots of pre-built code, much of which it ships with, downloading etc. Packages cannot download things on their own outside of the packaging system on Gentoo. Which is why in part gradle, maven, and others cannot be used. There are many Java projects packaged on Gentoo, that do not use the projects build system. Unfortunately they are just not compatible at this time, and may never be compatible.

I do not want to use sources that are embedded in jars. That is just the only place that code can be found. If it was in a public repo, I would use that instead. The only other choice is to use pre-built jars, and that goes against Gentoo policies. If the code is a dependency of an open source project, it would be nice to have that code available as well. Seems it is just embedded for other purposes.

There are many jars used in the BND build system, that cannot be found else where. I did find them on a maven and another general Java jar site. But there is no official upstream. It seems that the BND project is the upstream for those jars. For that reason I would think even the BND project might care about dependencies. As for the source code within them. I guess I need to get in touch with Peter Kriens. It is just really confusing. Hopefully you can start to understand the dilemma from another perspective.

I could see about asking Peter Kriens to release the code. Though in a way that was done already embedded in the jars. The code is already out there. I get what your saying but its really strange, as it seems this code is needed for bnd. The reason it builds now is its using the pre-built code. Which is hosted within the BND project with no other clear upstream, binary or source.

Why I was saying might need to be another issue about the code in the jars. It is really hard to tell what is BND and what is not. When the only place things can be found, those jars, the code in them, is in the BND repo. BND even starts with aQute all over the place. So it is really confusing to figure out what is what.

[wltjr]

If I got in touch with Peter Kriens I think it would be to ask to release the code, and it seems it would go under the BND project with other code. Possible reducing duplication, as its clear this code was fixed else where. But I am not sure where else the code would go. If Peter Kriens release the code, who would become the upstream/maintainer? I do not want to maintain it. Just need it to compile and build. :)

What if someday BND cannot used the pre-built binary for what ever reason? I know that is completely hypothetical and not likely. It would be nice as an active project if any and all dependencies can be found and built from source. Or at least have some official upstream. There are some really strange dependencies of BND.

[pkriens]

I already moved some of the code to bnd because it needed to integrate with the proxy handling. However, the interfaces are still coming from jpm. I am ok to also add the remaining code to the bnd workspace but I guess we should maybe then move jpm to a separate project. Wdyt BJ?

[bjhargrave]

I already moved some of the code to bnd because it needed to integrate with the proxy handling. However, the interfaces are still coming from jpm. I am ok to also add the remaining code to the bnd workspace but I guess we should maybe then move jpm to a separate project. Wdyt BJ?

That is fine with me.

However I am not sure even with all the source code in the bnd repo that you can build bnd properly in the Gentoo build. bnd is self-hosted. That is, to properly build bnd, you need bnd to build itself. It is sort of like a C compiler written in C. Obviously the original versions of bnd could be built without bnd. But then we began to use prior versions of bnd to build bnd. Now we use the immediate prior build of bnd to build bnd. Without using bnd to built bnd, you will have to generate a lot of information manually (manifest) and also package each jar manually (with content included as directed by the bnd.bnd files).

[wltjr]

I did not run into any bootstrapping issues with bnd. What you describe is part of the problem with maven and gradle on Gentoo. Also openjdk, as it needs a JDK to compile java code, so you have to bootstrap that. Like icedtea 7, 8, require icedtea 6.

The issue I have run into is it seems bnd.jar has lots of stuff in that jar, libg, repository, etc. Things I have split into multiple packages. I need to see about making a full bnd on Gentoo somehow. Not sure if having that stuff on the classpath is enough, or if it needs to be within the bnd jar file itself. If all that stuff needs to be in the one bnd.jar, it will be pretty difficult on Gentoo.

I get what your are saying though. BND adds "extra stuff" to projects, and I will need to run bnd against itself to add that information. I think that is in part what I am missing, as the resulting bnd from the Gentoo ebuild is not really usable yet. Having issues building jgroups which uses bnd. One issue I ran into was missing properties files, for ant bnd task the taskdef.properties, but I have a feeling I am missing lots of other stuff that is needed.

Now that I have the base bnd building, I have to go about adding the rest of the stuff so it is usable and complete. Though likely will be different than official bnd, and the resulting jar that the default build system generates.

[fhuberts]

Are you guys seriously splitting jars? If so, then you don't understand java; jars are deployment units.

If you split them, then suddenly every program using a split jar will have to be patched to fix its classpath.

If the author packaged lots of packages in a jar, he must have a reason.

[bjhargrave]

I am not sure of the ultimate value of attempting to recreate the bnd build system. It will be a constantly moving target as bnd and its build is changed from release to release.

However, were I a potential user of bnd, I would be more likely to trust the version built by the bnd team than the version built by you for Gentoo. Please make sure the bnd you build is clearly marked to distinguish it from the official bnd builds.

Good luck on this Sisyphean task.

[wltjr]

Splitting things up is the default way on Gentoo. Though for some things it might require all to be in one project. I very much understand Java, there just has been a different way of packaging and building Java applications on Gentoo for some time. This predates maven, gradle, ivy, and others. Gentoo unlike most operating systems has a proper package dependency system. If others did as well, I think how Java has evolved would be very different.

What if another project say uses libg, but not the rest of BND. You rather a polluted classpath with extra stuff on there not needed? The idea is package it to the smallest pieces so dependencies can use the bare minimum. It does create work for things that need it all.

On Gentoo the classpath issue is dealth with rather easily. Packages record their build and runtime classpath so that others can use that as needed. Granted this might be much larger than normal, as it will have many jars, vs a few.

I am not trying to re-create the build system, but simply package this on Gentoo. Unfortunately the build system and gentoo are not compatible. Many projects build systems change. This is part of the job of packaging things. There might be the option of providing a binary package, but those are hardly ideal. That would be me building it using the default build system outside of Gentoo. Then I take the result and do an "install" only package of the binary I made. Which makes it a binary package, vs from source. But those are not ideal for many reasons.

I understand some may want to use official vs a version in a distribution. Though not sure about support, as there could be reasons why the official would not support an abnormal runtime environment like Gentoo. However if someone wanted to vet the code. Having it all from source, provides means for such. Not to mention if they need to patch or customized in some way. Also you end up with newer code across the board. In my BND all code, including dependencies can be set to a particular version of java. I am compiling most of bnd with 1.7 source/target. Also using the latest version of everything.

I am building biz.aQute.respostory against the latest jetty for example, 9.3.7, not the version in bnd, 7.6.3. Also same for snakeyaml and some other things. There could be bugs fixed in snakeyaml and jetty for example. Thus while not official can be newer in various ways, built against newer dependencies and newer class file versions/target. Which can all be beneficial to the end user. Not to mention upstream, as you all can update your dependencies as well. We can test what works and does not, what might need to be fixed/updated, etc. Also makes it easier for support, as Gentoo can support its bnd package. Rather than the bnd project having to support bnd on Gentoo, which might be very different than bnd on other operating systems.

As for marking official vs unofficial. Its pretty clear when things are being compiled and built in Gentoo. the result was not built by upstream. Not sure anyone would confuse the package in Gentoo with an officially built bnd package. This would be no different for bnd than any other java application packaged on Gentoo.

[wltjr]

FYI I was able to build and use bnd, both the ant task and the wrap from within ant work just fine. That was the main need as some packages build and have bnd update the manifest in the resulting jar. This works perfectly! :100: But there could be issues with other uses of BND, cli, maven, plugins etc. That stuff will be addressed as needed. It was mostly needed for the manifest aspect per below.

Almost would be nice to OSGi'fy and run all jars on Gentoo through BND. Though not sure its practical due to all the build deps of BND. Still might be a possibility in the future. Thanks for all your work on BND. Have faith in the code more than the build system!

From JGroups jar built via JGroups ebuild, also being used in recent Guava ebuild.

wlt@wlt /tmp/jgroups $ cat META-INF/MANIFEST.MF
Manifest-Version: 1.0
Ant-Version: Apache Ant 1.9.1
Created-By: 1.8.0_74 (Oracle Corporation)
Bnd-LastModified: 1456101151584
Bundle-Description: build.xml file for JGroups. Needs Ant (jakarta.apa
che.org) to run
Bundle-DocURL: http://www.jboss.org
Bundle-License: http://www.opensource.org/licenses/lgpl-2.1.php
Bundle-ManifestVersion: 2
Bundle-Name: JGroups
Bundle-RequiredExecutionEnvironment: J2SE-1.5
Bundle-SymbolicName: org.jgroups
Bundle-Vendor: JBoss, a division of Red Hat
Bundle-Version: 3.2.7.Final
Export-Package: org.jgroups;version="3.2.7.Final";uses:="org.jgroups.a
nnotations,org.jgroups.conf,org.jgroups.logging,org.jgroups.stack,org
.jgroups.util,org.w3c.dom",org.jgroups.annotations;version="3.2.7.Fin
al";uses:="org.jgroups.conf",org.jgroups.auth;version="3.2.7.Final";u
ses:="javax.crypto,org.jgroups,org.jgroups.annotations,org.jgroups.lo gging,org.jgroups.protocols,org.jgroups.util",org.jgroups.blocks;vers ion="3.2.7.Final";uses:="javax.management,org.jgroups,org.jgroups.ann otations,org.jgroups.logging,org.jgroups.protocols,org.jgroups.stack, org.jgroups.util",org.jgroups.blocks.atomic;version="3.2.7.Final";use s:="org.jgroups,org.jgroups.protocols",org.jgroups.blocks.executor;ve rsion="3.2.7.Final";uses:="org.jgroups,org.jgroups.logging,org.jgroup s.protocols,org.jgroups.util",org.jgroups.blocks.locking;version="3.2 .7.Final";uses:="org.jgroups,org.jgroups.protocols,org.jgroups.util", org.jgroups.blocks.mux;version="3.2.7.Final";uses:="org.jgroups,org.j groups.blocks,org.jgroups.logging,org.jgroups.stack,org.jgroups.util" ,org.jgroups.client;version="3.2.7.Final";uses:="javax.net,org.jgroup s.annotations,org.jgroups.logging",org.jgroups.conf;version="3.2.7.Fi nal";uses:="org.jgroups,org.jgroups.logging,org.jgroups.util,org.w3c. dom",org.jgroups.demos;version="3.2.7.Final";uses:="javax.crypto,java x.management,javax.swing,org.jgroups,org.jgroups.annotations,org.jgro ups.blocks,org.jgroups.blocks.executor,org.jgroups.blocks.locking,org .jgroups.client,org.jgroups.util",org.jgroups.demos.applets;version=" 3.2.7.Final",org.jgroups.demos.wb;version="3.2.7.Final";uses:="org.jg roups,org.jgroups.blocks",org.jgroups.jmx;version="3.2.7.Final";uses: ="javax.management,org.jgroups,org.jgroups.stack",org.jgroups.logging ;version="3.2.7.Final",org.jgroups.protocols;version="3.2.7.Final";us es:="bsh,javax.crypto,javax.sql,org.jgroups,org.jgroups.annotations,o rg.jgroups.auth,org.jgroups.blocks,org.jgroups.blocks.atomic,org.jgro ups.blocks.executor,org.jgroups.blocks.locking,org.jgroups.conf,org.j groups.logging,org.jgroups.protocols.pbcast,org.jgroups.stack,org.jgr oups.util",org.jgroups.protocols.pbcast;version="3.2.7.Final";uses:=" org.jgroups,org.jgroups.annotations,org.jgroups.conf,org.jgroups.logg ing,org.jgroups.protocols,org.jgroups.stack,org.jgroups.util",org.jgr oups.protocols.relay;version="3.2.7.Final";uses:="org.jgroups,org.jgr oups.annotations,org.jgroups.logging,org.jgroups.protocols.relay.conf ig,org.jgroups.stack,org.jgroups.util",org.jgroups.protocols.relay.co nfig;version="3.2.7.Final";uses:="org.jgroups,org.jgroups.stack,org.w 3c.dom",org.jgroups.protocols.tom;version="3.2.7.Final";uses:="org.jg roups,org.jgroups.annotations,org.jgroups.stack,org.jgroups.util",org .jgroups.stack;version="3.2.7.Final";uses:="org.jgroups,org.jgroups.a nnotations,org.jgroups.conf,org.jgroups.logging,org.jgroups.protocols ,org.jgroups.util",org.jgroups.tests.perf;version="3.2.7.Final";uses: ="org.jgroups,org.jgroups.logging,org.jgroups.util",org.jgroups.util; version="3.2.7.Final";uses:="javax.management,org.jgroups,org.jgroups .annotations,org.jgroups.auth,org.jgroups.blocks,org.jgroups.logging, org.jgroups.protocols,org.jgroups.protocols.pbcast,org.jgroups.stack" Implementation-Version: 3.2.7.Final Import-Package: bsh;resolution:=optional,javax.crypto,javax.crypto.spe c,javax.management,javax.naming,javax.net,javax.net.ssl,javax.sql,jav ax.swing,javax.swing.table,javax.xml.namespace,javax.xml.parsers,java x.xml.transform,javax.xml.transform.dom,javax.xml.transform.stream,ja vax.xml.xpath,org.apache.log4j,org.jgroups;version="[3.2,4)",org.jgro ups.annotations;version="[3.2,4)",org.jgroups.auth;version="[3.2,4)", org.jgroups.blocks;version="[3.2,4)",org.jgroups.blocks.atomic;versio n="[3.2,4)",org.jgroups.blocks.executor;version="[3.2,4)",org.jgroups .blocks.locking;version="[3.2,4)",org.jgroups.blocks.mux;version="[3. 2,4)",org.jgroups.client;version="[3.2,4)",org.jgroups.conf;version=" [3.2,4)",org.jgroups.jmx;version="[3.2,4)",org.jgroups.logging;versio n="[3.2,4)",org.jgroups.protocols;version="[3.2,4)",org.jgroups.proto cols.pbcast;version="[3.2,4)",org.jgroups.protocols.relay;version="[3 .2,4)",org.jgroups.protocols.relay.config;version="[3.2,4)",org.jgrou ps.stack;version="[3.2,4)",org.jgroups.util;version="[3.2,4)",org.w3c .dom,org.xml.sax,org.xml.sax.helpers Main-Class: org.jgroups.Version Require-Capability: osgi.ee;filter:="(&(osgi.ee=JavaSE)(version=1.7))" Tool: Bnd-3.1.0

[rotty3000]

On Thu, Feb 25, 2016 at 1:27 PM, William L. Thomson Jr. < notifications@github.com> wrote:

Bundle-RequiredExecutionEnvironment: J2SE-1.5

There seems to be a discrepancy with the execution env header and the require capability header. You may want to check that. I don't think the bundle will resolve the way it is.

can someone confirm my assumption?

Raymond Augé http://www.liferay.com/web/raymond.auge/profile (@rotty3000) Senior Software Architect Liferay, Inc. http://www.liferay.com (@Liferay) Board Member & EEG Co-Chair, OSGi Alliance http://osgi.org (@OSGiAlliance)

[wltjr]

I was just looking at that stuff. I believe its the defaults in JGroups, I need to modify. https://github.com/belaban/JGroups/blob/JGroups_3_2_7_Final/conf/jgroups.bnd

They have changed for 3.3.0 alpha and in master. I was trying to get BND built and working. There is much to improve wrt to OSGi and Gentoo. I for sure need to do more to correct the manifest so the bundle can be used properly.

[bjhargrave]

can someone confirm my assumption?

I will require java 7. The cap for a java 7 will also offer 5 so the BREE will be satisfied.

[bjhargrave]

I was just looking at that stuff. I believe its the defaults in JGroups, I need to modify. https://github.com/belaban/JGroups/blob/JGroups_3_2_7_Final/conf/jgroups.bnd

You should remove the BREE. bnd will calculate the osgi.ee requirement which replaces BREE.

[wltjr]

Ok that is fairly easy, I can remove it with some sed vs patch. Not sure if I can use BND with the existing Gentoo OSGi stuff, think it should be possible.

If you scroll down a bit the OSGi bundle stuff should stand out https://github.com/gentoo/gentoo/blob/master/eclass/java-osgi.eclass

I might make a generic gentoo.bnd file or something that can be used.