search

nativelibs4java / JNAerator

JNAerator: native bindings generator for JNA / BridJ / Node.js
http://jnaerator.googlecode.com
504 stars 108 forks source link

jnaerator does not create methods/subclasses (0.10 and 0.11) #33

Open ochafik opened 9 years ago

ochafik commented 9 years ago

From @timk on July 2, 2012 12:2

Hi.

I am trying to jnearate libav-0.8.3 but compared to the library FFMpeg built here there is a lot of classes and methods missing. And using that library or build script it fails from missing classes.

When building with 'jnaerator-0.10-shaded.jar' or 'jnaerator-0.11-SNAPSHOT-shaded.jar' the only files created is the library files as 'AVCodecLibrary.java'.

See this for reference for what should be: https://github.com/ochafik/nativelibs4java/tree/master/libraries/FFMpeg/src/main/java/com/nativelibs4java/ffmpeg/avcodec

I also found the issue https://github.com/ochafik/nativelibs4java/issues/288 where some of this is discussed but it isn't really a bug for this particular problem so I thought I'd open this to help pin the problem down.

My output

$ java -jar jnaerator-0.11-SNAPSHOT-shaded.jar config.jnaerator
Auto-configuring parser...
2012-jul-02 13:35:41 com.ochafik.lang.jnaerator.JNAeratorConfigUtils getProp
INFO: [environment] JNAERATOR_INCLUDE_PATH=.:/usr/include:/usr/include/c++/4.6.3:/usr/include/c++/4.6.3/tr1
2012-jul-02 13:35:41 com.ochafik.lang.jnaerator.JNAeratorConfigUtils getProp
INFO: [environment] JNAERATOR_FRAMEWORKS_PATH=/System/Library/Frameworks/CoreServices.framework/Versions/Current/Frameworks:/System/Library/Frameworks/ApplicationServices.framework/Versions/Current/Frameworks:/System/Library/Frameworks:/Library/Frameworks:/Local/Library/Frameworks/:/home/timk/Library/Frameworks
os.arch = amd64
2012-jul-02 13:35:41 com.ochafik.lang.jnaerator.JNAerator jnaerate
INFO: Include path : 
    /home/timk/sources/libav-0.8.3
    /usr/include/x86_64-linux-gnu
    /usr/include/linux
    /usr/include/c++/4.6/x86_64-linux-gnu
    /usr/include/dirac
    /usr/include/schroedinger-1.0
    /usr/x86_64-w64-mingw32/include
    .
    /usr/include
    /usr/include/c++/4.6.3
    /usr/include/c++/4.6.3/tr1
Parsing native headers...
/home/timk/sources/libav-0.8.3/libavcodec/vda.h:28:0: error: File not found: VideoDecodeAcceleration/VDADecoder.h in /home/timk/sources/libav-0.8.3 /usr/include/x86_64-linux-gnu /usr/include/linux /usr/include/c++/4.6/x86_64-linux-gnu /usr/include/dirac /usr/include/schroedinger-1.0 /usr/x86_64-w64-mingw32/include . /usr/include /usr/include/c++/4.6.3 /usr/include/c++/4.6.3/tr1
/usr/x86_64-w64-mingw32/include/winnt.h:219:4: error: #error Must define a target architecture.
Writing preprocessor output to '_jnaerator.preprocessed.c'
Writing preprocessor macros to '_jnaerator.macros.cpp'
line 1:1 no viable alternative at character '\'
line 1:4 no viable alternative at character '\'
line 1:14 mismatched character '<EOF>' expecting '"'
Normalizing parsed code...
Writing raw parsed sources to '_jnaerator.rawParsed.cpp'
Writing normalized parsed sources to '_jnaerator.normalizedParsed.cpp'
Generating libraries...
Overwriting file '_jnaerator.choices'
Compiling JNAerated files...
Generating out.jar
#
# SUCCESS: JNAeration completed !
# Output mode is 'Jar(JAR with bindings only)
#
# => '/home/timk/workspace/Transcoder/lib/out.jar'
#

config.jnaerator

-I /home/timk/sources/libav-0.8.3
-I /usr/include/x86_64-linux-gnu/
-I /usr/include/linux/
-I /usr/include/c++/4.6/x86_64-linux-gnu/
-I /usr/include/dirac
-I /usr/include/schroedinger-1.0
-I /usr/x86_64-w64-mingw32/include

"/usr/lib/x86_64-linux-gnu/libavcodec.so" 
"/usr/lib/x86_64-linux-gnu/libavdevice.so" 
"/usr/lib/x86_64-linux-gnu/libavfilter.so" 
"/usr/lib/x86_64-linux-gnu/libavformat.so" 
"/usr/lib/x86_64-linux-gnu/libavutil.so" 
"/usr/lib/x86_64-linux-gnu/libpostproc.so"  
"/usr/lib/x86_64-linux-gnu/libswscale.so"  

-library avcodec    /home/timk/sources/libav-0.8.3/libavcodec/internal.h 
-library avcodec    /home/timk/sources/libav-0.8.3/libavcodec/*.h 
-library avcodec    /home/timk/sources/libav-0.8.3/libavcodec/avcodec.h 
-library avdevice   /home/timk/sources/libav-0.8.3/libavdevice/*.h
-library avdevice   /home/timk/sources/libav-0.8.3/libavdevice/avdevice.h
-library avfilter   /home/timk/sources/libav-0.8.3/libavfilter/*.h
-library avfilter   /home/timk/sources/libav-0.8.3/libavfilter/avfilter.h
-library avformat   /home/timk/sources/libav-0.8.3/libavformat/*.h
-library avformat   /home/timk/sources/libav-0.8.3/libavformat/avformat.h
-library avutil     /home/timk/sources/libav-0.8.3/libavutil/opt.h
-library avutil     /home/timk/sources/libav-0.8.3/libavutil/*.h
-library avutil     /home/timk/sources/libav-0.8.3/libavutil/avutil.h
-library postproc   /home/timk/sources/libav-0.8.3/libpostproc/*.h
-library postproc   /home/timk/sources/libav-0.8.3/libpostproc/postprocess.h
-library swscale    /home/timk/sources/libav-0.8.3/libswscale/*.h
-library swscale    /home/timk/sources/libav-0.8.3/libswscale/swscale.h

-runtime Bridj
-reification
-scalaStructSetters

-root com.nativelibs4java.ffmpeg

-mode Jar
//-noJar -noComp
-arch linux_x64
//-o src/main/java
-o /home/timk/workspace/Transcoder/lib
-v
-f

Example from AVCodecLibrary.java: http://pastebin.com/CZu3jgD5

Copied from original issue: ochafik/nativelibs4java#324

ochafik commented 9 years ago

Hi @timk ,

Thanks for your detailed report!

I'll look into this issue asap, in the meanwhile could you please try with the -parseChunks option and attach the files generated with -v (rawParsed.cpp, errors.txt or other files that keep changing names with each version :-S https://code.google.com/p/jnaerator/wiki/TroubleShootingJNAeration )

Cheers

ochafik commented 9 years ago

From @timk on July 2, 2012 13:24

With '-parseChunks', the parsing failed and produced a lot more output information! It seames like the methods, classes and more files is found.

However the parsing failes and this time produces no '.java'-files at all (without '-parseChunks' the library files was created).

I added all the output on gist since it was a lot of data, included the output and called it 'jnaerator output', there it seams to have why it crashed. https://gist.github.com/3033185

Some parts of the output:

...

Now parsing 2261 slices
Parsing failed : java.util.concurrent.TimeoutException
Parsing failed : java.util.concurrent.ExecutionException: com.ochafik.lang.jnaerator.JNAeratorParser$ParseError: Failed to parse because of java.lang.NullPointerException
Parsing failed : java.util.concurrent.ExecutionException: com.ochafik.lang.jnaerator.JNAeratorParser$ParseError: Failed to parse because of java.lang.NullPointerException
Parsing failed : java.util.concurrent.ExecutionException: com.ochafik.lang.jnaerator.JNAeratorParser$ParseError: Failed to parse because of java.lang.NullPointerException
Normalizing parsed code...

...

Guessing the below stats is Java problems after the parsing..?

...

931 problems (72 errors, 859 warnings)

...

And the crash:

...

com.ochafik.lang.compiler.CompilerUtils$CompilationError: Error in ? at line 2687, col 42 :
    Type mismatch: cannot convert from double to float
Error in ? at line 2688, col 43 :
    Type mismatch: cannot convert from double to float

...

    at com.ochafik.lang.jnaerator.JNAerator$1$2.setFinished(JNAerator.java:692)
    at com.ochafik.lang.jnaerator.JNAerator.jnaerate(JNAerator.java:973)
    at com.ochafik.lang.jnaerator.JNAerator$1.finished(JNAerator.java:727)
    at com.ochafik.lang.jnaerator.JNAeratorCommandLineArgs$ArgsParser.parse(JNAeratorCommandLineArgs.java:135)
    at com.ochafik.lang.jnaerator.JNAerator.main(JNAerator.java:238)
    at com.ochafik.lang.jnaerator.JNAerator.main(JNAerator.java:211)
#
# ERROR: JNAeration failed !

...
ochafik commented 9 years ago

From @sequoiar on August 24, 2014 4:25

same issue with jnaerator-0.11-shaded.jar, that could not generate java code for C function.

ochafik commented 9 years ago

From @Jeansen on October 14, 2014 11:27

I've got the same problem. Trying to run jnaerator against pdh.h results in a lot of files with version 0.9.9 but with 0.11 I get only the main wrapper and without any methods. Running with -parseChunks gives me:


Okt 14, 2014 11:18:12 AM com.ochafik.admin.visualstudio.VisualStudioUtils getProp
INFO: [environment] ProgramFiles(x86)=C:\Program Files (x86)
Okt 14, 2014 11:18:12 AM com.ochafik.admin.visualstudio.VisualStudioUtils getProp
INFO: [environment] VISUAL_STUDIO_HOME=C:\Program Files (x86)\Microsoft Visual Studio 9.0
Okt 14, 2014 11:18:12 AM com.ochafik.admin.visualstudio.VisualStudioUtils getProp
INFO: [environment] ProgramFiles(x86)=C:\Program Files (x86)
Okt 14, 2014 11:18:12 AM com.ochafik.admin.visualstudio.VisualStudioUtils getProp
INFO: [environment] WINDOWS_SDK_HOME=C:\Program Files (x86)\Microsoft SDKs\Windows\v6.0A
Okt 14, 2014 11:18:12 AM com.ochafik.admin.visualstudio.VisualStudioUtils getProp
INFO: [environment] VISUAL_STUDIO_INCLUDES=C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\include;C:\Program Files (x86)\Microsoft SDKs\Windows\v6.0A\Include
Auto-configuring parser...
Okt 14, 2014 11:18:12 AM com.ochafik.lang.jnaerator.JNAeratorConfigUtils getProp
INFO: [environment] JNAERATOR_INCLUDE_PATH=C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\include;C:\Program Files (x86)\Microsoft SDKs\Windows\v6.0A\Include;.
Okt 14, 2014 11:18:12 AM com.ochafik.lang.jnaerator.JNAeratorConfigUtils getProp
INFO: [environment] JNAERATOR_FRAMEWORKS_PATH=/System/Library/Frameworks/CoreServices.framework/Versions/Current/Frameworks;/System/Library/Frameworks/ApplicationServices.framework
/Versions/Current/Frameworks;/System/Library/Frameworks;/Library/Frameworks;/Local/Library/Frameworks/;C:\Users\mlautenbach/Library/Frameworks
os.arch = amd64
Okt 14, 2014 11:18:12 AM com.ochafik.lang.jnaerator.JNAerator jnaerate
INFO: Include path :
        C:\Program Files (x86)\Windows Kits\8.1\Include\um
        C:\Program Files (x86)\Windows Kits\8.1\Include\shared
        C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\include
        C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\include
        C:\Program Files (x86)\Microsoft SDKs\Windows\v6.0A\Include
        .
Parsing native headers...
C:\Program Files (x86)\Windows Kits\8.1\Include\um\winnt.h:18:20: error: #error "No Target Architecture"
C:\Program Files (x86)\Windows Kits\8.1\Include\um\winnt.h:18:2070: error: #error Must define a target architecture.
Writing preprocessor output to '_jnaerator.preprocessed.c'
Writing preprocessor macros to '_jnaerator.macros.cpp'
Now trying to parse sources slice by slice.
Now parsing 345 slices
Normalizing parsed code...
Writing raw parsed sources to '_jnaerator.rawParsed.cpp'
Writing normalized parsed sources to '_jnaerator.normalizedParsed.cpp'
Generating libraries...
Generating PROCESSOR_NUMBER.java
Generating GROUP_AFFINITY.java
Generating FLOAT128.java
Generating LARGE_INTEGER.java
Generating ULARGE_INTEGER.java
Generating LUID.java
Generating LIST_ENTRY.java
Generating SINGLE_LIST_ENTRY.java
Generating LIST_ENTRY32.java
Generating LIST_ENTRY64.java
Generating OBJECTID.java
Struct with no field : _ENUM_FLAG_INTEGER_FOR_SIZE<1 >
Generating _ENUM_FLAG_INTEGER_FOR_SIZE<1 >.java
java.io.FileNotFoundException: .\test\_ENUM_FLAG_INTEGER_FOR_SIZE<1 >.java (The filename, directory name, or volume label syntax is incorrect)
        at com.ochafik.lang.jnaerator.JNAerator$1$2.setFinished(JNAerator.java:699)
        at com.ochafik.lang.jnaerator.JNAerator.jnaerate(JNAerator.java:980)
        at com.ochafik.lang.jnaerator.JNAerator$1.finished(JNAerator.java:734)
        at com.ochafik.lang.jnaerator.JNAeratorCommandLineArgs$ArgsParser.parse(JNAeratorCommandLineArgs.java:135)
        at com.ochafik.lang.jnaerator.JNAerator.main(JNAerator.java:239)
        at com.ochafik.lang.jnaerator.JNAerator.main(JNAerator.java:212)
#
# ERROR: JNAeration failed !
#
#       java.io.FileNotFoundException: .\test\_ENUM_FLAG_INTEGER_FOR_SIZE<1 >.java (The filename, directory name, or volume label syntax is incorrect)
#

Version 0.12 SNAPSHOT gives me:

Okt 14, 2014 11:27:03 AM com.ochafik.admin.visualstudio.VisualStudioUtils getProp
INFO: [environment] ProgramFiles(x86)=C:\Program Files (x86)
Okt 14, 2014 11:27:03 AM com.ochafik.admin.visualstudio.VisualStudioUtils getProp
INFO: [environment] VISUAL_STUDIO_HOME=C:\Program Files (x86)\Microsoft Visual Studio 9.0
Okt 14, 2014 11:27:03 AM com.ochafik.admin.visualstudio.VisualStudioUtils getProp
INFO: [environment] ProgramFiles(x86)=C:\Program Files (x86)
Okt 14, 2014 11:27:03 AM com.ochafik.admin.visualstudio.VisualStudioUtils getProp
INFO: [environment] WINDOWS_SDK_HOME=C:\Program Files (x86)\Microsoft SDKs\Windows\v6.0A
Okt 14, 2014 11:27:03 AM com.ochafik.admin.visualstudio.VisualStudioUtils getProp
INFO: [environment] VISUAL_STUDIO_INCLUDES=C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\include;C:\Program Files (x86)\Microsoft SDKs\Windows\v6.0A\Include
Auto-configuring parser...
Okt 14, 2014 11:27:04 AM com.ochafik.lang.jnaerator.JNAeratorConfigUtils getProp
INFO: [environment] JNAERATOR_INCLUDE_PATH=C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\include;C:\Program Files (x86)\Microsoft SDKs\Windows\v6.0A\Include;.
Okt 14, 2014 11:27:04 AM com.ochafik.lang.jnaerator.JNAeratorConfigUtils getProp
INFO: [environment] JNAERATOR_FRAMEWORKS_PATH=/System/Library/Frameworks/CoreServices.framework/Versions/Current/Frameworks;/System/Library/Frameworks/ApplicationServices.framework
/Versions/Current/Frameworks;/System/Library/Frameworks;/Library/Frameworks;/Local/Library/Frameworks/;C:\Users\mlautenbach/Library/Frameworks
os.arch = amd64
Okt 14, 2014 11:27:04 AM com.ochafik.lang.jnaerator.JNAerator jnaerate
INFO: Include path :
        C:\Program Files (x86)\Windows Kits\8.1\Include\um
        C:\Program Files (x86)\Windows Kits\8.1\Include\shared
        C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\include
        C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\include
        C:\Program Files (x86)\Microsoft SDKs\Windows\v6.0A\Include
        .
Parsing native headers...
C:\Program Files (x86)\Windows Kits\8.1\Include\um\winnt.h:18:20: error: #error "No Target Architecture"
C:\Program Files (x86)\Windows Kits\8.1\Include\um\winnt.h:18:2070: error: #error Must define a target architecture.
Writing preprocessor output to '_jnaerator.preprocessed.c'
Writing preprocessor macros to '_jnaerator.macros.cpp'
Now parsing 345 slices
Normalizing parsed code...
Writing raw parsed sources to '_jnaerator.rawParsed.cpp'
Writing normalized parsed sources to '_jnaerator.normalizedParsed.cpp'
Generating libraries...
Generating PROCESSOR_NUMBER.java
Generating GROUP_AFFINITY.java
Generating FLOAT128.java
Generating LARGE_INTEGER.java
Generating ULARGE_INTEGER.java
Generating LUID.java
Generating LIST_ENTRY.java
Generating SINGLE_LIST_ENTRY.java
Generating LIST_ENTRY32.java
Generating LIST_ENTRY64.java
Generating OBJECTID.java
Struct with no field : _ENUM_FLAG_INTEGER_FOR_SIZE<1 >
Generating _ENUM_FLAG_INTEGER_FOR_SIZE<1 >.java
java.io.FileNotFoundException: .\test\_ENUM_FLAG_INTEGER_FOR_SIZE<1 >.java (The filename, directory name, or volume label syntax is incorrect)
        at java.io.FileOutputStream.open(Native Method)
        at java.io.FileOutputStream.<init>(Unknown Source)
        at java.io.FileOutputStream.<init>(Unknown Source)
        at java.io.PrintWriter.<init>(Unknown Source)
        at com.ochafik.lang.jnaerator.JNAerator$2.<init>(JNAerator.java:872)
        at com.ochafik.lang.jnaerator.JNAerator.newFileWriter(JNAerator.java:872)
        at com.ochafik.lang.jnaerator.JNAerator.newFileWriter(JNAerator.java:856)
        at com.ochafik.lang.jnaerator.JNAerator$5.getSourceWriter(JNAerator.java:946)
        at com.ochafik.lang.jnaerator.JNAerator.getSourceWriter(JNAerator.java:839)
        at com.ochafik.lang.jnaerator.JNAerator$6.getSourceWriter(JNAerator.java:954)
        at com.ochafik.lang.jnaerator.ClassOutputter.getClassSourceWriter(ClassOutputter.java:29)
        at com.ochafik.lang.jnaerator.DeclarationsConverter.outputConvertedStruct(DeclarationsConverter.java:651)
        at com.ochafik.lang.jnaerator.DeclarationsConverter.convertStructs(DeclarationsConverter.java:688)
        at com.ochafik.lang.jnaerator.DeclarationsConverter.fillLibraryMapping(DeclarationsConverter.java:909)
        at com.ochafik.lang.jnaerator.JNADeclarationsConverter.generateLibraryFiles(JNADeclarationsConverter.java:1147)
        at com.ochafik.lang.jnaerator.JNAerator.generateLibraryFiles(JNAerator.java:1483)
        at com.ochafik.lang.jnaerator.JNAerator.jnaerationCore(JNAerator.java:1455)
        at com.ochafik.lang.jnaerator.JNAerator.jnaerate(JNAerator.java:1008)
        at com.ochafik.lang.jnaerator.JNAerator$1.finished(JNAerator.java:776)
        at com.ochafik.lang.jnaerator.JNAeratorCommandLineArgs$ArgsParser.parse(JNAeratorCommandLineArgs.java:152)
        at com.ochafik.lang.jnaerator.JNAerator.main(JNAerator.java:224)
        at com.ochafik.lang.jnaerator.JNAerator.main(JNAerator.java:197)
java.io.FileNotFoundException: .\test\_ENUM_FLAG_INTEGER_FOR_SIZE<1 >.java (The filename, directory name, or volume label syntax is incorrect)
        at com.ochafik.lang.jnaerator.JNAerator$1$2.setFinished(JNAerator.java:743)
        at com.ochafik.lang.jnaerator.JNAerator.jnaerate(JNAerator.java:1056)
        at com.ochafik.lang.jnaerator.JNAerator$1.finished(JNAerator.java:776)
        at com.ochafik.lang.jnaerator.JNAeratorCommandLineArgs$ArgsParser.parse(JNAeratorCommandLineArgs.java:152)
        at com.ochafik.lang.jnaerator.JNAerator.main(JNAerator.java:224)
        at com.ochafik.lang.jnaerator.JNAerator.main(JNAerator.java:197)
#
# ERROR: JNAeration failed !
#
#       java.io.FileNotFoundException: .\test\_ENUM_FLAG_INTEGER_FOR_SIZE<1 >.java (The filename, directory name, or volume label syntax is incorrect)
#

Different to version 0.11 with 0.12-SNAPSHOT there are some classes which got generated (but not all) and the wrapper is missing.

raner commented 6 years ago

I just ran into a similar problem: JNAerator produced a bunch of static fields corresponding to #defines, but not a single method. When I ran with the -v option I noticed the following: the _jnaerator.preprocessed.c file contained a number of unexpanded macros that did not have any corresponding definition in the _jnaerator.macros.cpp file. Though I did not see any parsing errors reported further downstream, the preprocessed code (as far as I looked at it) clearly was not valid C/C++ code due to the unexpanded macro names being inserted in all sorts of places in the source code. To fix my problem, I added -Dmacro name= options to config.jnaerator, which removed the unexpanded macros and resulted in proper Java code generation. The spurious macros actually were defined (though I had to hunt across many #ifdefs and #includes), but somehow JNAerator's preprocessing did not pick up the correct values somewhere along the way...