Closed GoogleCodeExporter closed 8 years ago
Ah yes that's a good idea. Right now, JavaCPP simply skips classes that contain
no native methods, so I could take advantage of that to let the user specify a
package name, instead of a class name, to process all classes in the package.
How does that sound?
Original comment by samuel.a...@gmail.com
on 29 Feb 2012 at 11:25
If it collects the classes recursively from the specified package, that sounds
perfect.
Original comment by adam.waldenberg@gmail.com
on 29 Feb 2012 at 12:44
I was thinking to include only the classes in that package, like an `import`
statement in Java source files. There's no way to import recursively packages,
so I think it would be confusing if JavaCPP would start doing that, no? Do you
create packages that often?
What would not be confusing though is that if the user specifies no classes and
no packages, it would process ALL the classes specified by the "-classpath"
option.
How does that sound?
Original comment by samuel.a...@gmail.com
on 29 Feb 2012 at 10:58
Hm. Maybe. But with the -classpath way it would have to go through quite a lot.
Also, in that case, what would it do with dependencies under the classpath that
already implement native bindings (but not with javacpp) for example?
Those would have native methods, but no Javacpp annotations. I guess it would
try to generate bindings for those also?
An yeah... Collecting recursively from specified packages, could be confusing.
Original comment by adam.waldenberg@gmail.com
on 1 Mar 2012 at 7:58
Ok, I've modified the Builder to accept package name (followed by ".*" like
import statements..) or to scan all classes when no class or package names are
specified. Let me know how it feels, thanks!
Original comment by samuel.a...@gmail.com
on 1 Mar 2012 at 8:34
Attachments:
I tested it, and it's a big improvement.
Specifying packages works great. Not specifying anything doesn't really "work",
as it requires you to provide all dependencies (including the dependencies of
dependencies) under -classpath (otherwise you get errors about missing classes).
However, it's probably a good thing that JavaCPP at least tries to generate
something even when no class or package is provided.
Original comment by adam.waldenberg@gmail.com
on 1 Mar 2012 at 9:32
Do you mean that classes that make use of JavaCPP do not have these
dependencies and that's why it works when specifying the right package
manually? Or does it look like some bug? If it's the former, I could simply
ignore those classes, maybe throw a warning, and keep processing...
Original comment by samuel.a...@gmail.com
on 1 Mar 2012 at 9:50
No.. It's simply that JavaCPP now tries to process the dependencies (classes)
of classes found in the classpath, so to speak. If those dependencies are not
specified in the classpath, it bails out, saying the dependency (class) could
not be found.
I guess you could ignore classes that can't be found. But then a warning
becomes important, because otherwise you might miss unwanted behaviour.
I hope that makes sense :)...
Original comment by adam.waldenberg@gmail.com
on 1 Mar 2012 at 10:54
But why is it that you do not need to specify those dependencies in the
classpath when specifying classes or packages as arguments?
I've modified the Builder so it does not abort when it cannot load a class,
assuming that it is of no interest to it, but still outputs a warning on the
console. Let me know how that works, thanks!
Original comment by samuel.a...@gmail.com
on 1 Mar 2012 at 11:07
Attachments:
Ah yes.. I'm compiling with Maven, so it handles all transitive dependencies
for me... So I only have to consider direct dependencies for each
project/sub-project.
I'm not sure, but there might probably be something in the Maven dependency
plugin
to get all dependencies as a list or something.
In any case... I tried the above on the physics layer in our engine, and I get
the following output:
Warning: Could not load class com.ejwa.dinja.utility.xml.XMLReader:
java.lang.NoClassDefFoundError: org/simpleframework/xml/Serializer
Generating source file:
/home/adamw/Projects/dinja-engine/physics/target/classes/lib/armeabi/jniBulletNa
tive.cpp
Building library file:
/home/adamw/Projects/dinja-engine/physics/target/classes/lib/armeabi/libjniBulle
tNative.so
/home/adamw/Archives/android-ndk-r7/toolchains/arm-linux-androideabi-4.4.3/prebu
ilt/linux-x86/bin/arm-linux-androideabi-g++
--sysroot=/home/adamw/Archives/android-ndk-r7/platforms/android-9/arch-arm/
-I/usr/lib/jvm/java-6-sun-1.6.0.26/include
-I/usr/lib/jvm/java-6-sun-1.6.0.26/include/linux
-I/home/adamw/Archives/android-ndk-r7/sources/cxx-stl/gnu-libstdc++/include/
-I/home/adamw/Archives/android-ndk-r7/sources/cxx-stl/gnu-libstdc++/libs/armeabi
/include/
-I/home/adamw/Archives/android-ndk-r7/platforms/android-5/arch-arm/usr/include/
-I/home/adamw/Projects/dinja-engine/physics/src/main/java/../cpp/
/home/adamw/Projects/dinja-engine/physics/target/classes/lib/armeabi/jniBulletNa
tive.cpp -march=armv5te -mtune=xscale -msoft-float
/home/adamw/Archives/android-ndk-r7/sources/cxx-stl/gnu-libstdc++/libs/armeabi/l
ibsupc++.a -Wl,-rpath,lib/ -DANDROID -ffunction-sections -funwind-tables
-fstack-protector -funswitch-loops -finline-limit=300 -Wall -O3 -nodefaultlibs
-fPIC -shared -Wl,--no-allow-shlib-undefined -s -o
/home/adamw/Projects/dinja-engine/physics/target/classes/lib/armeabi/libjniBulle
tNative.so
-L/home/adamw/Archives/android-ndk-r7/sources/cxx-stl/gnu-libstdc++/libs/armeabi
/ -L/home/adamw/Archives/android-ndk-r7/platforms/android-5/arch-arm/usr/lib/
-L/home/adamw/Projects/dinja-engine/physics/target/local/armeabi/ -lbullet
-llog -lgnustl_static -lgcc -ldl -lz -lm -lc
/home/adamw/Projects/dinja-engine/physics/target/classes/lib/armeabi/jniBulletNa
tive.cpp: In function 'void Java_java_awt_Color_initIDs(JNIEnv*, _jclass*)':
/home/adamw/Projects/dinja-engine/physics/target/classes/lib/armeabi/jniBulletNa
tive.cpp:2299: error: 'initIDs' was not declared in this scope
So it skips org/simpleframework/xml/Serializer which is a transitive dependency
from the utility project.
After that i'm not sure what happens, but it seems to have included
java.awt.Color, which has no native methods in it.
Original comment by adam.waldenberg@gmail.com
on 2 Mar 2012 at 9:12
java.awt.Color does have a native initIDs() method :)
As argument to "javacpp.jar -classpath", have you tried to specify the
directory of your project build files /only/, without include any other
dependencies?
Original comment by samuel.a...@gmail.com
on 2 Mar 2012 at 9:20
Oh BTW, the thing is, JavaCPP will search for classes only within the path we
specify for /its/ classpath option. We should specify the dependencies that we
don't want to process in Java's classpath, e.g.:
java -cp /my/dependencies.jar:javacpp.jar com.googlecode.javacpp.Builder -cp my/build/dir
Original comment by samuel.a...@gmail.com
on 2 Mar 2012 at 9:28
Ah... I know what the problem is then... We have our own java.awt.Color,
because Android doesn't offer any awt classess.. So I guess there is a clash
there.
Original comment by adam.waldenberg@gmail.com
on 2 Mar 2012 at 9:32
I guess its behaving as expected then... The clash above is a real corner case
that mostly wont occur anyway.
Original comment by adam.waldenberg@gmail.com
on 2 Mar 2012 at 9:40
Thinking about it, maybe adding an -r flag for recursing packages would be a
better idea...
Or an even better idea... Maybe being able to specify something like this:
com.company.package.* ; Includes all classes in package
com.company.package.** ; Includes all classes in package recursively.
Original comment by adam.waldenberg@gmail.com
on 2 Mar 2012 at 9:47
Do you know of any Java tools that actually do something like that?
BTW, what problem are you trying to solve? How many packages do you have like
that?
Original comment by samuel.a...@gmail.com
on 2 Mar 2012 at 10:30
Well, it's a common way to denote recursion in different tools. Together with
java packages... I'm not sure. But it's at least a common enough operator for
people to understand what it does I believe.
I'm not sure I understand your question, but I can't get the solution with the
classpath to work. Specifically, it's because of the clash from our utility jar
(java.awt.Color), however, there are dependencies from utility also being used
in classes which should be processed by JavaCPP, so I can't remove that jar
from JavaCPP's classpath.
Specifying packages separately works, but you still need to update the build
file if you add or remove packages. Still, it's better than before.
Just having to specify something like (in our case)
<argument>com.ejwa.dinja.physics.**</argument> would have been really nice.
Original comment by adam.waldenberg@gmail.com
on 2 Mar 2012 at 11:19
Maybe it should be possible to get this to work...
I tried executing JavaCPP without supplying any packages or classes and moved
all dependencies but the builddir to java -cp. This is what I get:
Warning: Could not load class com.ejwa.dinja.physics.math.PhysicsTransform:
java.lang.NoClassDefFoundError: com/ejwa/dinja/utility/pool/Poolable
Warning: Could not load class com.ejwa.dinja.physics.math.PhysicsMatrix3:
java.lang.NoClassDefFoundError: com/ejwa/dinja/utility/pool/Poolable
Warning: Could not load class com.ejwa.dinja.physics.math.PhysicsQuaternion:
java.lang.NoClassDefFoundError: com/ejwa/dinja/utility/pool/Poolable
Warning: Could not load class com.ejwa.dinja.physics.math.PhysicsVector3:
java.lang.NoClassDefFoundError: com/ejwa/dinja/utility/pool/Poolable
Generating source file:
/home/chaozer/Projects/dinja-engine/physics/target/classes/lib/armeabi/jniBullet
Native.cpp
Exception in thread "main" java.lang.NoClassDefFoundError:
com/ejwa/dinja/utility/pool/Poolable
So now it's Generator.java that is bailing out.
Original comment by adam.waldenberg@gmail.com
on 2 Mar 2012 at 11:56
Yeah, maybe it's not a good idea to fool around in packages with missing
dependencies...
What I'd like to know is, you obviously don't want to process
com.ejwa.dinja.physics.math.*, but maybe you want to process
com.ejwa.dinja.native.this.* and com.ejwa.dinja.native.that.*, etc. how many
packages do you have with native methods for JavaCPP? I guess it's possible to
have native methods pop up here and there randomly whenever we need to
"optimize" something when the JIT sucks too much, is this what is happening
here?
I found one other Java tool that uses the ** glob with package names:
http://codemesh.com/products/vfour/tutorial/basics/L_03_import_builtin_java_type
s.html
It sounds reasonable.. I mean, someone entering com.package.* would get the
expected behavior, while adding an extra * gets us recursion, sounds good.
Eventually I would see something like javac doing the job of JavaCPP (similar
to how Microsoft compiles their C++/CLI "assemblies"), and have it output a
.class and .so files, or one .jar and one .so, or whatever. But this possible
future isn't affected if Builder supports something like **, so I'm not against
it :) And if it can help the initial adoption stages of this kind of tool, why
not?
Original comment by samuel.a...@gmail.com
on 2 Mar 2012 at 12:17
When it comes to the Dinja Engine physics layer... Pretty much all packages
have native methods... Including math. It tries to follow the structure of the
Bullet library and how classes are laid out there. This way, we get small
compact files for each class that we have bindings for. Right now, we are
running JavaCPP like this, using the Builder.java from above:
<argument>com.ejwa.dinja.physics.collision.*</argument>
<argument>com.ejwa.dinja.physics.collision.dispatch.*</argument>
<argument>com.ejwa.dinja.physics.collision.shape.*</argument>
<argument>com.ejwa.dinja.physics.dynamics.*</argument>
<argument>com.ejwa.dinja.physics.dynamics.solver.*</argument>
<argument>com.ejwa.dinja.physics.library.*</argument>
<argument>com.ejwa.dinja.physics.math.*</argument>
Which works very well... With support for ** that would become a one-liner.
Theoretically, (and stating the obvious) if we wanted to exclude for example
math (and had support for **),
we could do something like:
<argument>com.ejwa.dinja.physics.collision.**</argument>
<argument>com.ejwa.dinja.physics.dynamics.**</argument>
<argument>com.ejwa.dinja.physics.library.*</argument>
Which also looks really nice if you ask me :).
Anyway.. I looked closer at what was happening in Generator.java with the above
error, and yeah, getDeclaredMethods(), is failing because of the missing
dependencies. I guess it might be fixable by modifying the class loader... But
it's starting to feel quite hairy and kind of like painting one self into a
corner :)...
Original comment by adam.waldenberg@gmail.com
on 2 Mar 2012 at 12:49
[deleted comment]
And yeah... If it wasn't for our java.awt.Color class that happens to be
emulating a class with a native method... The solution with the classpath
should have worked. So it might still be a feature to keep.
Original comment by adam.waldenberg@gmail.com
on 2 Mar 2012 at 1:01
Ok, I've added support for the .** glob in Loader and modified Generator to
skip over (with a warning) classes that fail with NoClassDefFoundError. Let me
know if it seems to be working both ways! thanks
Original comment by samuel.a...@gmail.com
on 2 Mar 2012 at 1:15
Attachments:
Oops, the Builder, sorry..
Original comment by samuel.a...@gmail.com
on 2 Mar 2012 at 1:15
Attachments:
Wow, that was fast... I'll try it at once :).
Original comment by adam.waldenberg@gmail.com
on 2 Mar 2012 at 1:16
Working like a charm! I tried:
<argument>com.ejwa.dinja.physics.collision.**</argument>
<argument>com.ejwa.dinja.physics.dynamics.**</argument>
<argument>com.ejwa.dinja.physics.library.*</argument>
and:
<argument>com.ejwa.dinja.physics.**</argument>
The first one correctly excludes the math package.
Great work :).
Original comment by adam.waldenberg@gmail.com
on 2 Mar 2012 at 1:26
If i disable the custom java.awt.Color class, not specifying anything also
works:
Warning: Could not load class com.ejwa.dinja.utility.xml.XMLReader:
java.lang.NoClassDefFoundError: org/simpleframework/xml/Serializer
Generating source file:
/home/adamw/Projects/dinja-engine/physics/target/classes/lib/armeabi/jniBulletNa
tive.cpp
Building library file:
/home/adamw/Projects/dinja-engine/physics/target/classes/lib/armeabi/libjniBulle
tNative.so
/home/adamw/Archives/android-ndk-r7/toolchains/arm-linux-androideabi-4.4.3/prebu
ilt/linux-x86/bin/arm-linux-androideabi-g++
--sysroot=/home/adamw/Archives/android-ndk-r7/platforms/android-9/arch-arm/
-I/usr/lib/jvm/java-6-sun-1.6.0.26/include
-I/usr/lib/jvm/java-6-sun-1.6.0.26/include/linux
-I/home/adamw/Archives/android-ndk-r7/sources/cxx-stl/gnu-libstdc++/include/
-I/home/adamw/Archives/android-ndk-r7/sources/cxx-stl/gnu-libstdc++/libs/armeabi
/include/
-I/home/adamw/Archives/android-ndk-r7/platforms/android-5/arch-arm/usr/include/
-I/home/adamw/Projects/dinja-engine/physics/src/main/java/../cpp/
/home/adamw/Projects/dinja-engine/physics/target/classes/lib/armeabi/jniBulletNa
tive.cpp -march=armv5te -mtune=xscale -msoft-float
/home/adamw/Archives/android-ndk-r7/sources/cxx-stl/gnu-libstdc++/libs/armeabi/l
ibsupc++.a -Wl,-rpath,lib/ -DANDROID -ffunction-sections -funwind-tables
-fstack-protector -funswitch-loops -finline-limit=300 -Wall -O3 -nodefaultlibs
-fPIC -shared -Wl,--no-allow-shlib-undefined -s -o
/home/adamw/Projects/dinja-engine/physics/target/classes/lib/armeabi/libjniBulle
tNative.so
-L/home/adamw/Archives/android-ndk-r7/sources/cxx-stl/gnu-libstdc++/libs/armeabi
/ -L/home/adamw/Archives/android-ndk-r7/platforms/android-5/arch-arm/usr/lib/
-L/home/adamw/Projects/dinja-engine/physics/target/local/armeabi/ -lbullet
-llog -lgnustl_static -lgcc -ldl -lz -lm -lc
Otherwise it tries to generate bindings for the java.awt.Color class in the
standard classes.
Anyway... It feels like the current solution should be able to cover most use
cases now.
Original comment by adam.waldenberg@gmail.com
on 2 Mar 2012 at 2:28
Oh, I almost forgot. I also tried moving dependencies up to java's class path
and I am only specifying the build directory as the classpath for javacpp.
In this case I get the following (not specifying any classes or packages to
generate from):
Warning: Could not load class com.ejwa.dinja.physics.math.PhysicsTransform:
java.lang.NoClassDefFoundError: com/ejwa/dinja/utility/pool/Poolable
Warning: Could not load class com.ejwa.dinja.physics.math.PhysicsMatrix3:
java.lang.NoClassDefFoundError: com/ejwa/dinja/utility/pool/Poolable
Warning: Could not load class com.ejwa.dinja.physics.math.PhysicsQuaternion:
java.lang.NoClassDefFoundError: com/ejwa/dinja/utility/pool/Poolable
Warning: Could not load class com.ejwa.dinja.physics.math.PhysicsVector3:
java.lang.NoClassDefFoundError: com/ejwa/dinja/utility/pool/Poolable
Generating source file:
/home/adamw/Projects/dinja-engine/physics/target/classes/lib/armeabi/jniBulletNa
tive.cpp
2012-mar-02 15:51:12 com.googlecode.javacpp.Generator doClasses
VARNING: Could not generate code for class
com.ejwa.dinja.physics.collision.shape.BoxShape
2012-mar-02 15:51:12 com.googlecode.javacpp.Generator doClasses
VARNING: Could not generate code for class
com.ejwa.dinja.physics.collision.shape.StaticPlaneShape
2012-mar-02 15:51:12 com.googlecode.javacpp.Generator doClasses
VARNING: Could not generate code for class
com.ejwa.dinja.physics.dynamics.IDynamicsWorld
2012-mar-02 15:51:12 com.googlecode.javacpp.Generator doClasses
VARNING: Could not generate code for class
com.ejwa.dinja.physics.dynamics.DiscreteDynamicsWorld
2012-mar-02 15:51:12 com.googlecode.javacpp.Generator doClasses
VARNING: Could not generate code for class
com.ejwa.dinja.physics.dynamics.RigidBodyConstructionInfo
2012-mar-02 15:51:12 com.googlecode.javacpp.Generator doClasses
VARNING: Could not generate code for class
com.ejwa.dinja.physics.math.MotionState
2012-mar-02 15:51:12 com.googlecode.javacpp.Generator doClasses
VARNING: Could not generate code for class
com.ejwa.dinja.physics.math.DefaultMotionState
2012-mar-02 15:51:12 com.googlecode.javacpp.Generator doClasses
VARNING: Could not generate code for class
com.ejwa.dinja.physics.collision.shape.BoxShape
2012-mar-02 15:51:12 com.googlecode.javacpp.Generator doClasses
VARNING: Could not generate code for class
com.ejwa.dinja.physics.collision.shape.StaticPlaneShape
2012-mar-02 15:51:12 com.googlecode.javacpp.Generator doClasses
VARNING: Could not generate code for class
com.ejwa.dinja.physics.dynamics.IDynamicsWorld
2012-mar-02 15:51:12 com.googlecode.javacpp.Generator doClasses
VARNING: Could not generate code for class
com.ejwa.dinja.physics.dynamics.DiscreteDynamicsWorld
2012-mar-02 15:51:12 com.googlecode.javacpp.Generator doClasses
VARNING: Could not generate code for class
com.ejwa.dinja.physics.dynamics.RigidBodyConstructionInfo
2012-mar-02 15:51:12 com.googlecode.javacpp.Generator doClasses
VARNING: Could not generate code for class
com.ejwa.dinja.physics.math.MotionState
2012-mar-02 15:51:12 com.googlecode.javacpp.Generator doClasses
VARNING: Could not generate code for class
com.ejwa.dinja.physics.math.DefaultMotionState
Building library file:
/home/adamw/Projects/dinja-engine/physics/target/classes/lib/armeabi/libjniBulle
tNative.so
/home/adamw/Archives/android-ndk-r7/toolchains/arm-linux-androideabi-4.4.3/prebu
ilt/linux-x86/bin/arm-linux-androideabi-g++
--sysroot=/home/adamw/Archives/android-ndk-r7/platforms/android-9/arch-arm/
-I/usr/lib/jvm/java-6-sun-1.6.0.26/include
-I/usr/lib/jvm/java-6-sun-1.6.0.26/include/linux
-I/home/adamw/Archives/android-ndk-r7/sources/cxx-stl/gnu-libstdc++/include/
-I/home/adamw/Archives/android-ndk-r7/sources/cxx-stl/gnu-libstdc++/libs/armeabi
/include/
-I/home/adamw/Archives/android-ndk-r7/platforms/android-5/arch-arm/usr/include/
-I/home/adamw/Projects/dinja-engine/physics/src/main/java/../cpp/
/home/adamw/Projects/dinja-engine/physics/target/classes/lib/armeabi/jniBulletNa
tive.cpp -march=armv5te -mtune=xscale -msoft-float
/home/adamw/Archives/android-ndk-r7/sources/cxx-stl/gnu-libstdc++/libs/armeabi/l
ibsupc++.a -Wl,-rpath,lib/ -DANDROID -ffunction-sections -funwind-tables
-fstack-protector -funswitch-loops -finline-limit=300 -Wall -O3 -nodefaultlibs
-fPIC -shared -Wl,--no-allow-shlib-undefined -s -o
/home/adamw/Projects/dinja-engine/physics/target/classes/lib/armeabi/libjniBulle
tNative.so
-L/home/adamw/Archives/android-ndk-r7/sources/cxx-stl/gnu-libstdc++/libs/armeabi
/ -L/home/adamw/Archives/android-ndk-r7/platforms/android-5/arch-arm/usr/lib/
-L/home/adamw/Projects/dinja-engine/physics/target/local/armeabi/ -lbullet
-llog -lgnustl_static -lgcc -ldl -lz -lm -lc
Which is correct behaviour I guess. All the classes where code generation is
failing have some kind of dependency on one of the classes reported at the top.
Original comment by adam.waldenberg@gmail.com
on 2 Mar 2012 at 3:00
Great, thanks for testing all of that. I've included those changes in the
latest release. It should all still be working as above...
Original comment by samuel.a...@gmail.com
on 3 Mar 2012 at 4:52
Original issue reported on code.google.com by
adam.waldenberg@gmail.com
on 29 Feb 2012 at 10:03