Package a maven version of Java bindings

[opencv-pushbot]

Transferred from http://code.opencv.org/issues/3097

|| Abhishek Kumar on 2013-06-17 16:00
|| Priority: Normal
|| Affected: None
|| Category: java bindings
|| Tracker: Feature
|| Difficulty: 
|| PR: 
|| Platform: None / None

Package a maven version of Java bindings

We need to get OpenCV bundled with native code as a maven dependency.

Ant build systems have long been moving into oblivion. Though not getting deprecated anytime soon, all non-trivial have been moving towards Maven. 
As such, availability of OpenCV as a maven dependency would satisfy many users while adding many new hobbyists to OpenCV user list.

History

Daniel Baggio on 2014-01-09 21:00

Hi,

I have a sample pom.xml file that shows how to create a Java OpenCV project with Maven at:

https://github.com/JavaOpenCVBook/code/tree/master/chapter1/maven-sample

I would be glad to help in providing an official way to do it. Basic need is to generate a native JNI dynamic link library for each of the main platforms (windows, linux, osx, 32 and 64 bit). If that is created, a maven project is straightfoward. 

Kind regards,
Daniel

Maurice Betzel on 2014-01-10 20:04

I have succesfully used the https://code.google.com/p/cmake-maven-project CMake-Maven plugin for cross platform deployments. Even have it running as a lib bundle under OSGi.

Thomas Weißschuh on 2014-03-12 21:21

+1

Daniil Osokin on 2014-03-17 07:41

Hi, guys! Please contribute this feature to OpenCV (code.opencv.org/projects/opencv/wiki/How_to_contribute). It will help a lot (and speed up the process in many times: OpenCV Jedis can efficiently communicate with you)!

Daniel Baggio on 2014-05-27 01:36

Actually, it doesn't have to do with the code, but it should address people who create the build itself.
With buildbot, I believe we should be able to get it working easily, but I don't know how to get in contact with people who has got to do with it.

Sergey Lisovoy on 2014-08-06 10:46

I create project on GitHub that build maven artifact that contains https://github.com/lis0x90/opencv-maven-bundle:
* opencv native (currently for linux64, win64 and win32)
* opencv java binding 
* loader (use hawtjni to unpack and load native library in runtime)

Feel free to clone, build and test!

Daniel Baggio on 2014-10-21 03:38

Seriously, this should really be seen as a top priority task for Java developers.
Others also required it here:
http://answers.opencv.org/question/30945/why-isnt-the-opencv-java-libraries-available-via/

Any updates on this feature?

Steven Puttemans on 2014-10-21 07:53

Hi Daniel,

As long as noone will actually take up the task and do it, then I am afraid it could take quite a long time since OpenCV developers are doing this in their spare time.
Are you up to supplying a PR for this yourself? Or contact the right people for it?

Steven

Daniel Baggio on 2014-11-12 23:04

Sorry, Steven, but what do you mean by PR?

Kind regards,
Daniel

Steven Puttemans on 2014-11-13 13:32

With a pull request I mean an adaptation done by you to the code base through the github interface.
More information can be found here: http://code.opencv.org/projects/opencv/wiki/How_to_contribute

Daniel Baggio on 2014-11-19 01:53

Thanks Steven.

Unfortunately, this can't be done with a pull request since this is part of the build process. If only could I get in touch with people who make OpenCV releases, this could be addressed more easily. It is just a matter of uploading jar files to maven repositories, that would make a huge difference for Java developers, since it could take away the part of generating binaries.

Stojan Pesov on 2015-03-04 08:42

I agree with Daniel, we are eager to see the jar put in maven from you guys officially. 
That will be great advantage for all Java developers.

Daniel Baggio on 2015-06-25 05:56

Please, has anyone seen this feature?

[GavinBeirne]

+1. This will make java development much simpler and can't be addressed with a pull request. I hope someone with influence in the release process sees/understands this issue eventually.

[bachi76]

+1. I bet the open cv adoption rate in the java world would be much higher if that painful setup process could be simplified by a maven repo like everything else does these days.

[dannyxyz22]

Well, in case anyone needs, I set up a repository and there's a 5 minute setup procedure for developing OpenCV for Java with Maven here: https://github.com/JavaOpenCVBook/code/wiki/Tutorial-1----Java-OpenCV-5-minute-setup Currently only windows working. If anyone from IOS can give me a help, it will be available for you as well.

[bachi76]

@dannyxyz22 Thanks, that's great, let's hope the OpenCV team will take over and integrate it into their build / dist process. The other main problem is that they deliver only binaries for Windows, not for Linux (or iOS). Trying to build OpenCV manually ends quickly in dependency hell. I really hope they start to deliver binaries for all major platforms, not just Windows and Android.

[dannyxyz22]

@bachi76 So do I :+1:

[Enet4]

I have kept a custom build of the Java bindings in a private Maven repository for a while, but it would have been much better if Maven releases were handled by the OpenCV team.

[mdindoffer]

It's been almost three years since opencv got java bindings. How the hell is the maven support still missing? This is ridiculous.

[dannyxyz22]

I think we should call the community attention here 😁 Em 28/10/2015 3:01 PM, "Leprechaunius" notifications@github.com escreveu:

It's been almost three years since opencv got java bindings. How the hell is the maven support still missing? This is ridiculous.

— Reply to this email directly or view it on GitHub https://github.com/Itseez/opencv/issues/4588#issuecomment-151909942.

[NickyWeber]

+1!

[samos123]

+1! I'm currently using https://github.com/bytedeco/javacv-examples as alternative.

[benmccann]

+1

[Rabrg]

+1

[ndtreviv]

+1

[dljsjr]

Howdy folks,

Maybe we can be of some assistance. We've been vendoring OpenCV's Java bindings on a locally hosted Nexus instance for a while, but we recently decided to go open-source so I'm trying to replace as much of our internally hosted stuff as possible with either things on Maven Central/jCenter or our own public-facing Bintray so folks don't have to hit our locally hosted server. Because our outgoing pipe is a dumpster fire, and our Internet uses a microwave link that dies whenever there's a thunderstorm. We live in Florida. It is not good.

Anywho. As I'm working on this cleanup, I decided I'd take a crack at this since there seems to be a need for it in the community. I'm new to the whole open-source thing and this definitely isn't ready for prime-time yet so I wanted to discuss it here before making a PR. It will almost definitely need to be cleaned up to address the fact that software engineering is a crapshoot of a craft and I've been phoning it in for the last 5 years of my life anyway.

I wanted to give a quick rundown of what I did, and get some feedback on what needs to be done to get the ball rolling to get some of this stuff landed in the upstream OpenCV repository. Also, if anybody out there wants to give me a hand, I won't complain.

You can check out the ihmc/maven_publish_using_gradle branch at our fork if you wanna follow along in the book: https://github.com/ihmcrobotics/opencv/tree/ihmc/maven_publish_using_gradle

First, I did not modify (much) the CMake/Ant build process. I added a gradle build on top of the CMake/Ant build, and in fact the Gradle build will invoke the CMake and Make/MSBuild processes if it needs to. But I did not delegate the responsibility of making the Maven artifacts on CMake, because I don't know CMake or Ant very well and it seems like it would be hard. I do know Gradle pretty well though. Plus Gradle is the de facto build system of Android.

The only changes I made to the CMake build process are:

1. Set "source" and "target" directives on the Ant javac task to 1.6. I use Java 8 so I wanted to make sure there was some backwards compatibility.
2. Do not include the sources in a single .jar. I'm basically treating it like the proper entry point in to the Java .jar is now the Gradle build, which will generate a separate .jar of sources with the sources classifier used by Maven. So you still get source attachments. But this can be changed; more on this later. This also means that the ant build now builds in a separate directory than the sources.
3. Set the file extension on the generated shared library object to .dylib instead of .so on OS X. Technically I should probably be generating a .jnilib as well but this can be done with a simple copy-paste. Shared libraries meant for JNI on OS X must be .jnilib on Apple Java 1.6 and standard OS X .dylib on OpenJDK/Oracle Java distributions 1.7 and higher.

The rest of the heavy lifting is done by the Gradle build. Lemme run down how it works:

1. Shell out to CMake from within the "build" directory using the default options from the documentation; cmake -DBUILD_SHARED_LIBS=OFF ... On Windows, I specify "Visual Studio 14 2015 Win64" because I only have access to VS2015 Community Edition and a 64 bit machine.
2. Shell out to make or MSBuild using the default arguments from the documentation
3. From here, there are 3 Gradle tasks that can be run:
   1. Build a .jar artifact that contains the .class files from the generated Java sources. This works by cracking open the .jar generated from the Ant build and pulling out the class files then making a new .jar out of it with the appropriate metadata. If we wanted to undo my changes to not bundling the sources, we could do so here by just cracking open the jar and only extracting the .class files then cracking it open again for the .java files when I make the source artifact.
   2. Build a platform-dependent .jar artifact that contains the shared libraries (.so/.dylib/`.dll)
   3. Build a .jar that contains the source attachments. This just bundles up the src directory generated by the Python scripts.

You can turn these artifacts in to Maven publications by using publishMavenOpenCV to make a Maven artifact with the source and class artifacts, or publishMavenOpenCVPlatformJar to make a Maven artifact with the native libraries in it. From there, it's dealer's choice for getting the artifacts online.

The branch this work was done on was based on and kept up-to-date-with upstream/master over the last two days, but I backported these changes to the 3.1.0 tag and made a 3.1.0-ihmc tag.

A few caveats:

Because we are a highly cross-platform organization (our lab is split almost evenly Windows/Mac/Linux), it's challenging to maintain build files with platform specific dependencies, and we also shy away from generating platform-specific artifacts for all of our products. This has led to a design decision in my implementation that may not be seen as favorable. This whole thing basically requires three machines to generate 4 artifacts, and all 4 of those artifacts are related. There are 3 artifacts for Windows, Linux, and OS X containing the shared libraries, and then the Java bindings artifact which depends on all three platform jars at the same time. We place more value on not having to maintain hacky build-files and OS-specific artifacts for every single one of our builds than we do on disk space and bandwidth. So this might be a sub-optimal solution for the general public.

This poses a problem if you want to, say, clone and publish to your local Maven repo ($HOME/.m2). You can get around this by editing the portion of the Gradle build that hand-writes the pom for the wrapper jar (build.gradle line 173) to exclude the OS's that you aren't currently using and running the publishToMavenLocal task.

We also like to work with Native Libraries being on the classpath instead of on disk. It makes configuration less complicated. We have a tool that we made (as does just about everybody) to handle this for us: https://github.com/ihmcrobotics/ihmc-native-library-loader. The native libraries are at org.opencv.<native library>, and IHMCNativeLibraryLoader has a nice syntax for loading native libraries that are on the classpath:

   static
   {
      NativeLibraryLoader.loadLibrary("org.opencv", Core.NATIVE_LIBRARY_NAME);
   }

I think it's obvious that this stuff isn't ready for prime-time but it definitely works and I think it could be a good start.

BONUS!

As of today, we are providing OpenCV 3.1.0 Java .jars based on the 3.1.0 tag available on our Bintray (not jcenter yet) for 64-bit Ubuntu 14.04, 64-bit OS X El Capitan, and 64-bit Win7/8/10: https://bintray.com/ihmcrobotics/maven-vendor/OpenCVJavaBindings/view. There are instructions on that site for how to add that Maven repo to your Maven/Gradle scripts and pull in the opencv artifact (like I mentioned above, you just need the opencv artifact, and all three native artifacts get pulled down with it). If there's interest we may try to backport to other versions.

Disclaimer: I make no promises about what feature sets they were compiled with or how well these things are going to work outside of Ubuntu 14.04, OS X El Capitan, and Win 7/8/10. That's all I have at my disposal for testing. The machines were pretty bare-bones, so they probably weren't compiled with a bunch of super dope features. But they definitely work. I also don't make promises about keeping up with bleeding edge; we'll probably keep up with stable releases as we see fit but I would really prefer that this starts a dialogue for getting this in to upstream instead of having IHMC Robotics maintain distribution for the Java bindings.

Finally, it should be noted that all commits referenced herein as well as this comment were written under duress and lack of sleep, and as such are not admissible in a court of law.

[albertcheng]

There are maven plugin to perform maven repository publishing. I have done that a few times. Anyone owning this item needs help?

[chzhong]

+1

Also I'd prefer OpenCV4Android on maven repository.

[elennaro]

+1 for maven

[sarehag]

+1

[haesleinhuepf]

+1

[deyles-zz]

+1

[johngodoi]

+1

[fmeola]

+1

[binarysubstrate]

+1

[mereshow]

+1

[arronmabrey]

+1

[lonewolf9277]

+1

[lonewolf9277]

Please do it. It will make our lives easier.

[alalek]

Some support for maven packaging is added by PR #6827 and these scripts are available in platforms/maven sub-folder.

[interstar]

+1 for a simple Maven packaging of OpenCV with native dependencies. (Ideally Linux 32!!!)

I'm trying to work with OpenCV from inside Clojure ... I followed this tutorial : http://docs.opencv.org/2.4/doc/tutorials/introduction/clojure_dev_intro/clojure_dev_intro.html (including making the native code and the jars myself and putting them in a local repo) but something isn't right because even the tutorial examples are throwing errors about not finding some of the classes. And anyway, the tutorial is taking me through OpenCV 2.4 which I assume is out of date.

If packaging all the dependencies into a single bundle for Java (and Clojure) programers can be done automatically as part of the build / release for OpenCV that would make many people's lives a lot easier. (And undoubtedly grow the OpenCV community.)

[khasburrahman]

+1

[iocanel]

+1

[mrmarbles]

+1

[severi]

+1

[scprotz]

+1

[ghanashyamvtatti]

+1

[liuyu81]

+1

[mbd-terma]

+1

[ricardojlrufino]

+1

[mdindoffer]

If anyone's interested, I've put together a short 2-part tutorial on how to create your own trivial maven repo with OpenCV and how to integrate it in your projects. Very simple, but effective for small projects: http://dindoffer.eu/2016/12/30/opencv-with-maven-part-1-create-your-own-maven-repo-wo-high-level-bullshit/

[Restioson]

+1

[serezhka]

+1

[shivasaxena]

+1

[martinspielmann]

+1

[SDMitchell]

+1

[jjYBdx4IL]

+1

The current solution (bintray) does not work, pros don't add unreliable dependencies to their projects. That's why there is maven central where published stuff cannot be deleted.

If you guys need something at maven central, you could try JavaCV bindings, but they are not the official ones and I'd prefer to use the latter.

[tmladek]

+1, please!

[yangguangxiadeyun]

+1

[rohanliston]

+1

[nachtfisch]

+1

[tslater]

Low priority ;-(