No simple way to compile and package (with Maven) from source

[GoogleCodeExporter]

Can you create instruction?

What I tried to do:

1) git clone https://code.google.com/p/javacpp/
cd javacpp/javacpp && mvn install

2) git clone https://code.google.com/p/javacv/
cd javacv/javacv && mvn package

Seems like missing dependecines:

package com.jogamp.common.os does not exist
package javax.media.opengl does not exist

and a lot of:

cannot find symbol
[ERROR] symbol  : class CLImage2d
[ERROR] location: class com.googlecode.javacv.ProCamTransformerCL

Original issue reported on code.google.com by kuznetsov.alexey on 13 May 2012 at 10:05

[GoogleCodeExporter]

Yes, I should add to the pom.xml file any dependencies that might reasonable be 
available in some Maven repository somewhere, thanks for reporting this. I 
think that is pretty much limited to JogAmp's JOCL and JOGL though, do you see 
any other?

For the rest of the native dependencies, we either need to install them 
manually or remove the source files that depend on them, manually, and then 
"mvn install" or whatever as mentioned in the README.txt file. I haven't 
thought of any better ways at the moment, have you?

Original comment by samuel.a...@gmail.com on 14 May 2012 at 2:36

[GoogleCodeExporter]

package com.jogamp.opencl
package javax.media.opengl.glu
package com.jogamp.opencl.gl
package javax.media.opengl

maven can build c/cpp code and you can build opengl librarys too. i havent did 
it yet, this what i'm going to investigate.

what I did: put all natives to the jar (libtorrent-mac.jar) install it to the 
local maven repo and unpack them with maven-natives-plugin on the package stage 
to be abble to run application without installing librarys to the system folder.

i dont know if it gonna help or not. This project is completely new, and i dont 
know architecture or the way you doing things.

for 'mvn install' (manual makefile, just to reach to goal, better if it done 
with automake and maven cpp)

http://code.google.com/p/mircle/source/browse/bindings/java/libtorrent/Makefile?
repo=libtorrent&name=libtorrent-0.16.0

for 'maven-nativedependencies-plugin'

http://code.google.com/p/mircle/source/browse/bindings/java/pom.xml?repo=libtorr
ent&name=libtorrent-0.16.0

Original comment by kuznetsov.alexey on 14 May 2012 at 7:43

Attachments:

• build.log

[GoogleCodeExporter]

What you are trying to do is kind of related to issue #146 I guess...

Original comment by samuel.a...@gmail.com on 14 May 2012 at 7:49

[GoogleCodeExporter]

I've added a new sort of Maven repository 
http://maven2.javacv.googlecode.com/git/ that we can use inside pom.xml files.

Is the sufficient for your needs? Or are you actually looking into recompiling 
JavaCV?

Original comment by samuel.a...@gmail.com on 27 May 2012 at 1:22

[GoogleCodeExporter]

the same errors.

im trying to use javacv in the simple way. produce working javacv by running 
mvn package. that is it.

Original comment by kuznetsov.alexey on 28 May 2012 at 8:48

[GoogleCodeExporter]

Well, to rebuild JavaCV you will need to install manually all the dependencies 
or remove source files you do not need, as instructed in the README.txt file. 
So this is not the "simple way".

Original comment by samuel.a...@gmail.com on 28 May 2012 at 1:15

[GoogleCodeExporter]

Original comment by samuel.a...@gmail.com on 1 Jun 2012 at 2:18

• Changed title: No simple way to compile and package (with Maven) from source
• Added labels: Type-Enhancement
• Removed labels: Type-Defect

[GoogleCodeExporter]

It'd be nice if you put all java cv dependencies that are not in any other 
repository(jogamp+opencl) into your http://maven2.javacv.googlecode.com/git/ so 
every1 could be able to build javacv from clean checkout without removing any 
classes.
Also seems like Pro* classes should be in another maven module may be javacv 
project should have 2 or more modules.. something like that

Original comment by Jkolo...@gmail.com on 2 Jun 2012 at 8:00

[GoogleCodeExporter]

Actually at least javacv/cpp should be separated from everything else.

Original comment by Jkolo...@gmail.com on 2 Jun 2012 at 8:03

[GoogleCodeExporter]

The problem is that none of the dependencies have Maven artifacts. Someone 
would have to make them all. Feel like creating and maintaining them?

Sure, JavaCV could be split and stuff, but it won't save much space. People 
that do care use ProGuard and that takes of that anyway.

Original comment by samuel.a...@gmail.com on 2 Jun 2012 at 9:21

[GoogleCodeExporter]

They don't have to have maven artifacts. You can any groupId and artifactId and 
then import them into your remote repo 
(http://maven2.javacv.googlecode.com/git/)
http://maven.apache.org/guides/mini/guide-3rd-party-jars-remote.html

And then everyone will be able to download the dependencies from here 
automatically

Original comment by Jkolo...@gmail.com on 2 Jun 2012 at 11:26

[GoogleCodeExporter]

None of the source code of these dependencies come in JAR files. Would you 
volunteer to "JARify" everything, and maintain it?

Original comment by samuel.a...@gmail.com on 3 Jun 2012 at 1:12

[GoogleCodeExporter]

Ok. I was able to build gluegen jogl and jocl for mac. Looks like importing is 
going to be a pain in the butt because it produces 20+ jar files only for mac. 
But I'll try)

Original comment by Jkolo...@gmail.com on 3 Jun 2012 at 5:41

[GoogleCodeExporter]

Ok, thanks, but JogAmp should not be our main concern at this moment. If we 
can't do it with OpenCV, the rest is pretty much pointless IMO.

Original comment by samuel.a...@gmail.com on 3 Jun 2012 at 5:48

[GoogleCodeExporter]

Got it. I'll try to do the same with opencv.

Original comment by Jkolo...@gmail.com on 3 Jun 2012 at 6:12

[GoogleCodeExporter]

Hello. I've jarifyed opencv 2.4.1 for mac unix and win. I hope this is what you 
wanted to get. however it still needs some mechanism to extract it at runtime.
Also there are some scripts to build opencv for unix and mac and another one 
that downloads all jogamp.
http://goo.gl/xWOaw
Let me know what you think.

Original comment by Jkolo...@gmail.com on 4 Jun 2012 at 9:56

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

The same in one file http://goo.gl/A4ry6

Original comment by Jkolo...@gmail.com on 4 Jun 2012 at 10:19

[GoogleCodeExporter]

That looks interesting. When you say "Unix" do you mean "Linux"? Obviously we 
would need something that can build for all three platforms, and with Maven, 
since that is the goal here I think.

As for runtime, JavaCPP can do it just fine by itself. We just need to put the 
binaries in the right package as explained on this Wiki page:
http://code.google.com/p/javacv/wiki/HowToMakeAnApplet

Original comment by samuel.a...@gmail.com on 5 Jun 2012 at 1:10

[GoogleCodeExporter]

Yes. I mean linux. 
fixed according to wiki http://goo.gl/xWOaw

Original comment by Jkolo...@gmail.com on 5 Jun 2012 at 7:12

[GoogleCodeExporter]

Great thanks, starting to look good! So, is there a way to get this all 
compiled and packaged with just "mvn package"?

Original comment by samuel.a...@gmail.com on 5 Jun 2012 at 7:16

[GoogleCodeExporter]

yes. maven-exec-plugin http://mojo.codehaus.org/exec-maven-plugin/ allows to 
execute any script.
Actually I don't thik we need to build opencv everytime  we do mvn package. We 
can just put these prebuilt jars into our maven repo set them as dependencies 
and let everyone download it.

Original comment by Jkolo...@gmail.com on 5 Jun 2012 at 7:21

[GoogleCodeExporter]

No we can't because different users will want versions build with different 
sets of options.

Original comment by samuel.a...@gmail.com on 5 Jun 2012 at 7:23

[GoogleCodeExporter]

You can do something like 
mvn deploy:deploy-file -DgroupId=com.googlecode.javacv \
  -DartifactId=opencv-native-macosx \
  -Dversion=2.4.1 \
  -Dpackaging=jar \
  -Dfile=<path-to-file> \
  -DrepositoryId=<id-to-map-on-server-section-of-settings.xml> \
  -Durl=http://maven2.javacv.googlecode.com/git/

to import any jar into your maven repo

Original comment by Jkolo...@gmail.com on 5 Jun 2012 at 7:24

[GoogleCodeExporter]

ok. then we can import jar built with sh or cmd into local repo 
http://maven.apache.org/guides/mini/guide-3rd-party-jars-local.html

Original comment by Jkolo...@gmail.com on 5 Jun 2012 at 7:30

[GoogleCodeExporter]

Yeah, but it's not very automatic. In particular, how is it better than 
MacPorts, apt-get or yum? Actually, that's how I see it. There are package 
managers for native libraries, usually, so it makes sense to use those.

For standalone package distribution to users, well, we need something too, but 
I am not sure what... I don't think Maven is the answer, but your idea with 
these scripts to create standalone JARs may be useful to developers, outside 
Maven. What do you think?

Original comment by samuel.a...@gmail.com on 5 Jun 2012 at 7:40

[GoogleCodeExporter]

IMO this library is only for developer. so it should be standalone. For java 
developers it's the way more comfortable to switch between versions using maven 
than something else. Maven is cross platform unlike macport, yum etc.
Maven would allow 1click build/deployment after clean checkout.
yeah cmake + compilers are required to build opencv, but they should be 
installed only once. 

Original comment by Jkolo...@gmail.com on 5 Jun 2012 at 7:54

[GoogleCodeExporter]

actually I think the best way is to provide a default set of libs in our maven 
and provide something like these scripts for experienced users who want to 
build their own version with a different configuration

Original comment by Jkolo...@gmail.com on 5 Jun 2012 at 8:00

[GoogleCodeExporter]

Just wanted to add my 2c:
As a developer, I just want to use JavaCV as a java library: I would prefer to 
be able to do so without having to compile OpenCV.  I do development on Linux, 
mac and windows - going through the build process for each platform isn't 
ideal.  I don't mind installing it the OpenCV, but it should be as painless as 
possible.

If you can have it work so that I just need to include the javacv dependency in 
my pom, I'd be pretty happy.
The guys at xuggle (video processing API for java) do it this way - they depend 
on some platform specific stuff too, and they just include it in the jar as far 
as I know.

Original comment by dannyw...@gmail.com on 5 Jun 2012 at 8:28

[GoogleCodeExporter]

I agree with danny. the only people who MUST build opencv and other stuff from 
source should be the developers of javacv. Everyone else should get jars just 
from maven repo as a dependency.
My only concern is how to extract and load native libs in this case.

Original comment by Jkolo...@gmail.com on 5 Jun 2012 at 8:36

[GoogleCodeExporter]

Of course it's easier if everything is done automatically on all platforms..

Ok, so, what would you do for users that need feature X or feature Y? I guess 
we could make sure to provide the most complete builds. Could you try to 
rebuild OpenCV with all the options enabled, include all dependent libraries, 
and see what that gives?  

But what if GCC happens to compile feature X much better with SSE4.1, but some 
users don't have a CPU that supports SSE4.1, what now?

And like I said, JavaCPP takes care of extracting and sort of helping to load 
the native library depending on the platform, no need to worry about that

Original comment by samuel.a...@gmail.com on 5 Jun 2012 at 8:50

[GoogleCodeExporter]

to extract librarys on dev machines you have to use maven natives plugin

      <plugin>
        <groupId>com.googlecode.mavennatives</groupId>
        <artifactId>maven-nativedependencies-plugin</artifactId>
        <version>0.0.6</version>
        <executions>
          <execution>
            <id>unpacknatives</id>
            <goals>
              <goal>copy</goal>
            </goals>
          </execution>
        </executions>
      </plugin>

to extract and run on user machines you have to use installer to put librarys 
next to the app jars. also i use shade to pull all deps into one jar:

      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-shade-plugin</artifactId>
        <version>1.6</version>
        <executions>
          <execution>
            <phase>package</phase>
            <goals>
              <goal>shade</goal>
            </goals>
            <configuration>
              <artifactSet>
                <excludes>
                  <exclude>*:*:*:natives*</exclude>
                </excludes>
              </artifactSet>
            </configuration>
          </execution>
        </executions>
      </plugin>

Original comment by kuznetsov.alexey on 5 Jun 2012 at 8:51

[GoogleCodeExporter]

and i think using self extracted libs is a bad practice. use installers instead.

Original comment by kuznetsov.alexey on 5 Jun 2012 at 8:51

[GoogleCodeExporter]

Oh, and I forgot to mention that libraries should NOT depend on recent versions 
of libgcc. The easiest way is probably to build on an old version of Linux.. 
but then that means we can't use the kung fu from recent versions of GCC.. so, 
what should we do?

Original comment by samuel.a...@gmail.com on 5 Jun 2012 at 8:54

[GoogleCodeExporter]

actually our opencv build should not have more or less features than javacv 
supports.. about compiler optimisations... we can make this dependency OPTIONAL 
and let the user compile and use it's own version. the same as open cv does. 
they provide a source and some prebuilt versions. if you don't want to do 
anything - use prebuilt, if you want something special like gpurelated 
functions - compile it youself. we should do the same. java cv comes with the 
default opencv build. want something special? -go compile it yourself and 
import into maven, add as a dependency instead of our.

Original comment by Jkolo...@gmail.com on 5 Jun 2012 at 9:20

[GoogleCodeExporter]

JavaCV can access pretty much all features of OpenCV except opencv_gpu, but 
that could change, and the precompiled Windows binaries do come with the GPU 
module compiled.

Well, if you do want to contribute some binaries that mostly work, that would 
be good I think, as long as you maintain them. :) I won't be adding anything to 
the repo until we can confirm that it works well enough for most users though. 
For example, there is no opencv-natives-linux-x86_64.jar, the libraries are not 
in the com/googlecode/javacv/cpp/<platform.name>/ package namespace, the Mac OS 
X binaries haven't been fixed with `install_name_tool`, some dependent 
libraries are missing, the DLL names for Windows are incorrect, and probably 
other things. Try to fix as much as you can, test them with the `Demo` class in 
the README.txt file, and let me know what that gives, thanks!

Original comment by samuel.a...@gmail.com on 5 Jun 2012 at 9:56

[GoogleCodeExporter]

@kuznetsov Sure, we can install the native binaries too. JavaCPP will work just 
fine with preextracted and installed libraries.

Original comment by samuel.a...@gmail.com on 5 Jun 2012 at 9:59

[GoogleCodeExporter]

just to be clear, i mean application installer.

if you distribute some app with javacv libraries, the developer it self should 
care how javacv natives appears next to his own app jars.

maven package, and all this talk should be only about developers live. extra 
work should be done by the developer. who preaparing his own app for users.

i presonnaly don't like to look for opensource librarys in the system paths. on 
most platform that is not common, and you have really high chanse not to find 
them. so you have to include it in separate installer (which is more work for 
the user) on package all in one.

after some digging i found that the best way over packing all in one jar is to 
provide app installer which put jar's and natives librarys into os depended and 
user chosen folders. self extracting native librarys create tmp folders and may 
cause confclits when you run different app with different natives versions. 
also here is no way to pack all natives to the one jar file.

since you should have prepare separate jars  to each platform, isnt it more 
clear for user to have installer?

Original comment by kuznetsov.alexey on 5 Jun 2012 at 10:13

[GoogleCodeExporter]

AFAIK, we can do whatever we want with an installer. JavaCV imposes no 
restrictions that I know of, do you?

Original comment by samuel.a...@gmail.com on 5 Jun 2012 at 12:06

[GoogleCodeExporter]

got a question about install_name_tool. In that script on howToMakeAnApplet  
page what should be the value of BAD_PATH? the path of *dylib after 
compilation? 

Original comment by Jkolo...@gmail.com on 7 Jun 2012 at 5:10

[GoogleCodeExporter]

We can check that with `otool -L`, but there might be more than one "bad 
path"...

Original comment by samuel.a...@gmail.com on 7 Jun 2012 at 5:14

[GoogleCodeExporter]

It shows this
    /Users/jkolobok/Cameras/builds/opencv/OpenCV-2.4.1/build/lib/libopencv_video.2.4.dylib (compatibility version 2.4.0, current version 2.4.1)
    /Users/jkolobok/Cameras/builds/opencv/OpenCV-2.4.1/build/lib/libopencv_core.2.4.dylib (compatibility version 2.4.0, current version 2.4.1)
    /Users/jkolobok/Cameras/builds/opencv/OpenCV-2.4.1/build/lib/libopencv_imgproc.2.4.dylib (compatibility version 2.4.0, current version 2.4.1)
    /usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 52.0.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 159.1.0)

Original comment by Jkolo...@gmail.com on 7 Jun 2012 at 5:18

[GoogleCodeExporter]

So, one of the bad paths is 
"/Users/jkolobok/Cameras/builds/opencv/OpenCV-2.4.1/build/lib"

Original comment by samuel.a...@gmail.com on 7 Jun 2012 at 5:20

[GoogleCodeExporter]

So the result should looke like this, right?

MyBookPro:lib jkolobok$ otool -L libopencv_video.dylib 
libopencv_video.dylib:
    @rpath/lib/libopencv_video.2.4.dylib (compatibility version 2.4.0, current version 2.4.1)
    @rpath/libopencv_core.2.4.dylib (compatibility version 2.4.0, current version 2.4.1)
    @rpath/libopencv_imgproc.2.4.dylib (compatibility version 2.4.0, current version 2.4.1)
    /usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 52.0.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 159.1.0)

Original comment by Jkolo...@gmail.com on 7 Jun 2012 at 5:47

[GoogleCodeExporter]

So "@rpath/lib" is another "bad path".. fix all that, until you don't see any 
of those.

Original comment by samuel.a...@gmail.com on 7 Jun 2012 at 6:17

[GoogleCodeExporter]

what's wrong with windows libs.. which libraries are missing? mingw and vc10 
bin contains different number of libraries. I used mingw for the jar... may be 
this is the problem...

Original comment by Jkolo...@gmail.com on 7 Jun 2012 at 7:12

[GoogleCodeExporter]

Well, remove your c:\opencv\ directory and test them with the Demo class.

Original comment by samuel.a...@gmail.com on 9 Jun 2012 at 1:37

[GoogleCodeExporter]

ok. I'm trying to make it running on mac. 
Here is the problem. 
MyBookPro:macosx-x86_64 jkolobok$ otool -L libjniopencv_core.dylib 
libjniopencv_core.dylib:
    /Users/saudet/NetBeansProjects/javacv/build/classes/com/googlecode/javacv/cpp/macosx-x86_64/libjniopencv_core.dylib (compatibility version 0.0.0, current version 0.0.0)
    lib/libopencv_core.2.4.dylib (compatibility version 2.4.0, current version 2.4.0)
    /usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.9.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 125.2.1)

As the result of this I get 
Exception in thread "main" java.lang.UnsatisfiedLinkError: 
/private/var/folders/41/5cdqxf752cb6r5t3nyv4ks0m0000gn/T/javacpp1339719652540564
000/libjniopencv_core.dylib:  Library not loaded: lib/libopencv_core.2.4.dylib  
 Referenced from: 
/private/var/folders/41/5cdqxf752cb6r5t3nyv4ks0m0000gn/T/javacpp1339719652540564
000/libjniopencv_core.dylib   Reason: image not found

Original comment by Jkolo...@gmail.com on 15 Jun 2012 at 12:30

[GoogleCodeExporter]

Not sure where you got that libjniopencv_core.dylib file, I get this output 
here:
libjniopencv_core.dylib:
    /Users/saudet/NetBeansProjects/javacv/build/classes/com/googlecode/javacv/cpp/macosx-x86_64/libjniopencv_core.dylib (compatibility version 0.0.0, current version 0.0.0)
    @rpath/libopencv_core.2.4.dylib (compatibility version 2.4.0, current version 2.4.0)
    /usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.9.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 125.2.1)

Original comment by samuel.a...@gmail.com on 15 Jun 2012 at 12:54

[GoogleCodeExporter]

took it from javacv-0.1-bin.zip in Downloads

Original comment by Jkolo...@gmail.com on 15 Jun 2012 at 9:46