Enable build on Windows

christianpaquin commented 4 years ago

The Makefile needs to be rewritten for Windows.

vsoftco commented 4 years ago

@christianpaquin Maven build instructions "almost" work. When I do

mvn package -P windows -Dliboqs.include.dir="C:\Users\vsoft\Documents\GitHub\liboqs\build\includea" -Dliboqs.lib.dir="C:\Users\vsoft\Documents\GitHub\liboqs\build\lib"

I'm getting a Maven crash when trying to compile the unit tests:

INFO] ------------------------------------------------------- [INFO] T E S T S [INFO] ------------------------------------------------------- [INFO] Running org.openquantumsafe.KEMTest

[WARNING] Corrupted STDOUT by directly writing to native stream in forked JVM 1. See FAQ web page and the dump file C:\Users\vsoft\Documents\GitHub\liboqs-java\target\surefire-reports\2020-04-22T16-34-01_279-jvmRun1.dumpstream [INFO] [INFO] Results: [INFO] [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0 [INFO] [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 2.895 s [INFO] Finished at: 2020-04-22T16:34:02-04:00 [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M4:test (default-test) on project liboqs-java: There are test failures. [ERROR] [ERROR] Please refer to C:\Users\vsoft\Documents\GitHub\liboqs-java\target\surefire-reports for the individual test results. [ERROR] Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream. [ERROR] The forked VM terminated without properly saying goodbye. VM crash or System.exit called? [ERROR] Command was cmd.exe /X /C "C:\jdk-14.0.1\bin\java -Xss10M -Djava.library.path=C:\Users\vsoft\Documents\GitHub\liboqs-java\target -jar C:\Users\vsoft\AppData\Local\Temp\surefire15027720064054488022\surefirebooter8247440137434394921.jar C:\Users\vsoft\AppData\Local\Temp\surefire15027720064054488022 2020-04-22T16-34-01_279-jvmRun1 surefire12066564270794004930tmp surefire_015709868472019424591tmp" [ERROR] Error occurred in starting fork, check output in log [ERROR] Process Exit Code: 1 [ERROR] Crashed tests: [ERROR] org.openquantumsafe.KEMTest [ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException: The forked VM terminated without properly saying goodbye. VM crash or System.exit called? [ERROR] Command was cmd.exe /X /C "C:\jdk-14.0.1\bin\java -Xss10M -Djava.library.path=C:\Users\vsoft\Documents\GitHub\liboqs-java\target -jar C:\Users\vsoft\AppData\Local\Temp\surefire15027720064054488022\surefirebooter8247440137434394921.jar C:\Users\vsoft\AppData\Local\Temp\surefire15027720064054488022 2020-04-22T16-34-01_279-jvmRun1 surefire12066564270794004930tmp surefire_015709868472019424591tmp" [ERROR] Error occurred in starting fork, check output in log [ERROR] Process Exit Code: 1 [ERROR] Crashed tests: [ERROR] org.openquantumsafe.KEMTest [ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:690) [ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:285) [ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:248) [ERROR] at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1217) [ERROR] at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:1063) [ERROR] at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:889) [ERROR] at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137) [ERROR] at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:210) [ERROR] at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:156) [ERROR] at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:148) [ERROR] at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117) [ERROR] at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81) [ERROR] at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:56) [ERROR] at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128) [ERROR] at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:305) [ERROR] at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:192) [ERROR] at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:105) [ERROR] at org.apache.maven.cli.MavenCli.execute(MavenCli.java:957) [ERROR] at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:289) [ERROR] at org.apache.maven.cli.MavenCli.main(MavenCli.java:193) [ERROR] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [ERROR] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [ERROR] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [ERROR] at java.base/java.lang.reflect.Method.invoke(Method.java:564) [ERROR] at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:282) [ERROR] at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:225) [ERROR] at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:406) [ERROR] at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:347) [ERROR] [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

vsoftco commented 4 years ago

@christianpaquin Also when trying to run the examples, I can compile fine, e.g.

javac -cp target/liboqs-java.jar examples/SigExample.java

works fine, but running with

java -Djava.library.path=target/ -cp target/liboqs-java.jar:examples/ SigExample

produces

Error: Could not find or load main class SigExample
Caused by: java.lang.ClassNotFoundException: SigExample

jimouris commented 4 years ago

@vsoftco If you compile with -Dmaven.test.skip=true does it compile?

@christianpaquin Also when trying to run the examples, I can compile fine, e.g.
javac -cp target/liboqs-java.jar examples/SigExample.java 
works fine, but running with
java -Djava.library.path=target/ -cp target/liboqs-java.jar:examples/ SigExample
produces
Error: Could not find or load main class SigExample
Caused by: j ava.lang.ClassNotFoundException: SigExample

Does the compile command generate a SigExample.class inside the examples directory?

It seems that the class separator for the classpath in Windows is ; instead of : (https://docs.oracle.com/javase/tutorial/essential/environment/paths.html). Does it work if you try:

java -Djava.library.path=target/ -cp target/liboqs-java.jar;examples/ SigExample

vsoftco commented 4 years ago

@jimouris Yes, without testing is fine. I think it's a maven/jdk issue. However, for the examples, you were right, now they run, but on my side crash after the printing the enabled/supported KEMs/Sigs:

Supported signatures:
DEFAULT DILITHIUM_2 DILITHIUM_3 DILITHIUM_4 Falcon-512 Falcon-1024 MQDSS-31-48 MQDSS-31-64 Rainbow-Ia-Classic Rainbow-Ia-Cyclic Rainbow-Ia-Cyclic-Compressed Rainbow-IIIc-Classic Rainbow-IIIc-Cyclic Rainbow-IIIc-Cyclic-Compressed Rainbow-Vc-Classic Rainbow-Vc-Cyclic Rainbow-Vc-Cyclic-Compressed SPHINCS+-Haraka-128f-robust SPHINCS+-Haraka-128f-simple SPHINCS+-Haraka-128s-robust SPHINCS+-Haraka-128s-simple SPHINCS+-Haraka-192f-robust SPHINCS+-Haraka-192f-simple SPHINCS+-Haraka-192s-robust SPHINCS+-Haraka-192s-simple SPHINCS+-Haraka-256f-robust SPHINCS+-Haraka-256f-simple SPHINCS+-Haraka-256s-robust SPHINCS+-Haraka-256s-simple SPHINCS+-SHA256-128f-robust SPHINCS+-SHA256-128f-simple SPHINCS+-SHA256-128s-robust SPHINCS+-SHA256-128s-simple SPHINCS+-SHA256-192f-robust SPHINCS+-SHA256-192f-simple SPHINCS+-SHA256-192s-robust SPHINCS+-SHA256-192s-simple SPHINCS+-SHA256-256f-robust SPHINCS+-SHA256-256f-simple SPHINCS+-SHA256-256s-robust SPHINCS+-SHA256-256s-simple SPHINCS+-SHAKE256-128f-robust SPHINCS+-SHAKE256-128f-simple SPHINCS+-SHAKE256-128s-robust SPHINCS+-SHAKE256-128s-simple SPHINCS+-SHAKE256-192f-robust SPHINCS+-SHAKE256-192f-simple SPHINCS+-SHAKE256-192s-robust SPHINCS+-SHAKE256-192s-simple SPHINCS+-SHAKE256-256f-robust SPHINCS+-SHAKE256-256f-simple SPHINCS+-SHAKE256-256s-robust SPHINCS+-SHAKE256-256s-simple picnic_L1_FS picnic_L1_UR picnic_L3_FS picnic_L3_UR picnic_L5_FS picnic_L5_UR picnic2_L1_FS picnic2_L3_FS picnic2_L5_FS qTesla-p-I qTesla-p-III

Enabled signatures:
DEFAULT DILITHIUM_2 DILITHIUM_3 DILITHIUM_4 Falcon-512 Falcon-1024 MQDSS-31-48 MQDSS-31-64 Rainbow-Ia-Classic Rainbow-Ia-Cyclic Rainbow-Ia-Cyclic-Compressed Rainbow-IIIc-Classic Rainbow-IIIc-Cyclic Rainbow-IIIc-Cyclic-Compressed Rainbow-Vc-Classic Rainbow-Vc-Cyclic Rainbow-Vc-Cyclic-Compressed SPHINCS+-Haraka-128f-robust SPHINCS+-Haraka-128f-simple SPHINCS+-Haraka-128s-robust SPHINCS+-Haraka-128s-simple SPHINCS+-Haraka-192f-robust SPHINCS+-Haraka-192f-simple SPHINCS+-Haraka-192s-robust SPHINCS+-Haraka-192s-simple SPHINCS+-Haraka-256f-robust SPHINCS+-Haraka-256f-simple SPHINCS+-Haraka-256s-robust SPHINCS+-Haraka-256s-simple SPHINCS+-SHA256-128f-robust SPHINCS+-SHA256-128f-simple SPHINCS+-SHA256-128s-robust SPHINCS+-SHA256-128s-simple SPHINCS+-SHA256-192f-robust SPHINCS+-SHA256-192f-simple SPHINCS+-SHA256-192s-robust SPHINCS+-SHA256-192s-simple SPHINCS+-SHA256-256f-robust SPHINCS+-SHA256-256f-simple SPHINCS+-SHA256-256s-robust SPHINCS+-SHA256-256s-simple SPHINCS+-SHAKE256-128f-robust SPHINCS+-SHAKE256-128f-simple SPHINCS+-SHAKE256-128s-robust SPHINCS+-SHAKE256-128s-simple SPHINCS+-SHAKE256-192f-robust SPHINCS+-SHAKE256-192f-simple SPHINCS+-SHAKE256-192s-robust SPHINCS+-SHAKE256-192s-simple SPHINCS+-SHAKE256-256f-robust SPHINCS+-SHAKE256-256f-simple SPHINCS+-SHAKE256-256s-robust SPHINCS+-SHAKE256-256s-simple picnic_L1_FS picnic_L1_UR picnic_L3_FS picnic_L3_UR picnic_L5_FS picnic_L5_UR picnic2_L1_FS picnic2_L3_FS picnic2_L5_FS qTesla-p-I qTesla-p-III

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00007ffb5759cf10, pid=12292, tid=25812
#
# JRE version: OpenJDK Runtime Environment (14.0.1+7) (build 14.0.1+7)
# Java VM: OpenJDK 64-Bit Server VM (14.0.1+7, mixed mode, sharing, tiered, compressed oops, g1 gc, windows-amd64)
# Problematic frame:
# V  [jvm.dll+0xcf10]
#
# No core dump will be written. Minidumps are not enabled by default on client versions of Windows
#
# An error report file with more information is saved as:
# C:\Users\vsoft\Documents\GitHub\liboqs-java\hs_err_pid12292.log
#
# If you would like to submit a bug report, please visit:
#   https://bugreport.java.com/bugreport/crash.jsp
#

jimouris commented 4 years ago

Okay, so compilation works for Windows. Most likely, the error that you face when you build (with tests) is a similar crash to the above that you mentioned that happens when maven runs the tests.

I haven't seen this error before. I'll have to find out a way to replicate this error on Ubuntu and then I'll try to resolve it.

jimouris commented 4 years ago

The same problem exists on Mac as well (but not on Ubuntu).

It seems that the crash occurs in the native code when it calls OQS_KEM_keypair.

I checked all three parameters and both the public and the private key have been allocated properly and their lengths are correct (length_public_key and length_secret_key from the OQS_KEM struct). Also, the OQS_KEM *kem points to a correct memory location (same as OQS_KEM *kem = OQS_KEM_new(str_native);).

I'll look more into it.

jimouris commented 4 years ago

I was able to resolve the issue on Mac by updating liboqs and installing it again. I probably had an older version. I am not sure what may have caused that issue but it now works for me.

@vsoftco Can you try to pull and update liboqs to see if it works as well?

vsoftco commented 4 years ago

On my Mac it worked fine, just the Windows had issues. Just pull it and compiled liboqs, however the rand API changed slightly, and cannot link

https://github.com/open-quantum-safe/liboqs/blob/faa7589e6f01c8d479462a45aa1cdeeb2667702d/src/common/rand/rand.h#L65

so you'd need to alter the corresponding line in the wrapper.

jimouris commented 4 years ago

@vsoftco Did it work after you pulled?

Okay, I updated the rand call to the new API (fd612fae2770ea21109bba954ad4357ae6ecc7ce) and added instructions to run on Windows (3cc75006c26b529229a309771597d51d118e0742).

vsoftco commented 4 years ago

Still crashing when building unit tests (build liboqs alone fine), and unfortunately still crashing KEM and Sigs. Rand works (prints the NIST-KAT) but complains it cannot switch to OpenSSL (disable this under Windows, as we don't support it). Under Mac and Linux it works fine.

christianpaquin commented 4 years ago

OK, I finally have some bandwidth to test the Windows maven build. I'm not getting as far, @vsoftco: when trying to build the windows package (-P windows):

mvn package -P windows  -Dliboqs.include.dir="c:\dev\github\liboqs\build\include" -Dliboqs.lib.dir="c:\dev\github\liboqs\build\lib"

I get an error that gcc isn't found (see error below). Do you notice something wrong? Shouldn't the windows package call cl automatically?

[INFO] --- native-maven-plugin:1.0-alpha-9:compile (generate-shared-oqs-library) @ liboqs-java ---
[INFO] cmd.exe /X /C "gcc -IC:\Program Files\Java\jdk-14/include -IC:\Program Files\Java\jdk-14/include/win32 -fPIC -Wall -O3 -Ic:\dev\github\liboqs\build\include -Ic:\dev\github\liboqs-java\src\main\c -Ic:\dev\github\liboqs-java\target\generated-sources\annotations -oc:\dev\github\liboqs-java\target\objs\handle.obj -c c:\dev\github\liboqs-java\src\main\c\handle.c"
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  3.184 s
[INFO] Finished at: 2020-05-07T13:59:36-04:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:native-maven-plugin:1.0-alpha-9:compile (generate-shared-oqs-library) on project liboqs-java: Error executing command line: Error while executing process. Cannot run program "gcc" (in directory "c:\dev\github\liboqs-java"): CreateProcess error=2, The system cannot find the file specified -> [Help 1]

Anything obvious to you?

jimouris commented 4 years ago

I didn't have a chance to setup OQS in Windows yet.

We haven't defined different native compilers in pom.xml, gcc is hardcoded for now. Can you try changing lines 129 and 138 in pom.xml and see if it works?

If that works, I can define different compilers for different OSes in the profiles section like it is for the shared jni library.

christianpaquin commented 4 years ago

Oh, I see. There are a few more things to tweak, as the Windows compiler options are not the same as gcc. I'm curious how @vsoftco packaged the wrapper on Windows.

vsoftco commented 4 years ago

@christianpaquin Sorry I missed somehow the notification... I had MSYS2 installed and gcc was in the path, so most likely that's how the gcc was invoked.

christianpaquin commented 4 years ago

Pushing to next version.

baentsch commented 3 years ago

Quick question: Does anyone presently work on this? I'd otherwise try to see how an AWS-Windows instance could aid me in giving this a shot...

christianpaquin commented 3 years ago

I’m not.

On Mar 8, 2021, at 07:04, Michael Baentsch notifications@github.com wrote:

Quick question: Does anyone presently work on this? I'd otherwise try to see how an AWS-Windows instance could aid me in giving this a shot...

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fopen-quantum-safe%2Fliboqs-java%2Fissues%2F1%23issuecomment-792710844&data=04%7C01%7Ccpaquin%40microsoft.com%7C4a94000969f14c2a7d6208d8e22a4f93%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637508018646608095%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=FCLowjwJ7SMpvX3r8Uef21Xe6jp%2FsKcdJmyXVtoTCb8%3D&reserved=0, or unsubscribehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAD36T5PEAV4Y3W2MHLBLV63TCS4MLANCNFSM4ME3YAYQ&data=04%7C01%7Ccpaquin%40microsoft.com%7C4a94000969f14c2a7d6208d8e22a4f93%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637508018646608095%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=NB0j6QWQpEApOga92Qsbcemo0tVhf%2Bx0pvQQrx41doQ%3D&reserved=0.

jimouris commented 3 years ago

@baentsch I am not either but would be happy to help if you run into any issues/questions with the code.

baentsch commented 3 years ago

The problems I ran into were of the "Windows" sort: I just don't know enough about the tooling on that platform (and have not used maven in a long time -- and never the mojo plugin). Whatever: After replacing gcc with cl (and installing a suitable JDK (minimum 10)), maven started to operate OK, but didn't want to link the DLL. Main problem seems to be the configuration pom.xml: After setting cl as compiler this was the command executed by maven:

[INFO] cmd.exe /X /C "cl -Ic:\Users\Administrator\jdk-10/include -Ic:\Users\Administrator\jdk-10/include/win32 -Ic:\Users\Administrator\git\liboqs\build\include -IC:\Users\Administrator\git\liboqs-java\src\main\c -IC:\Users\Administrator\git\liboqs-java\target\generated-sources\annotations -o C:\Users\Administrator\git\liboqs-java\target\objs\Signature.obj -c C:\Users\Administrator\git\liboqs-java\src\main\c\Signature.c"
Microsoft (R) C/C++ Optimizing Compiler Version 19.28.29910 for x64
Signature.c
Copyright (C) Microsoft Corporation.  All rights reserved.

cl : Command line warning D9035 : option 'o' has been deprecated and will be removed in a future release

--> It seems the native build plugin doesn't detect that it's supposed to operate a MSVC compiler and use -Fo instead of the Unix -o. It seems to compile, but the object files don't wind up where they should. Do you know how this can be changed (@jimouris , assuming you know how to utilize Mojo)?

baentsch commented 3 years ago

PS: For anyone interested: This is now set up for "development" in an AWS Windows instance. Those of us without a local Win machine/VM could actually jointly use/share this to "trial" things on Windows: Anyone interested? We'd need to share a password for RDP access. OK for this purpose?

jimouris commented 3 years ago

It seems that the native plugin supports compiler/linker options and they mention -Fo for the msvc compiler.

I think that we need to overwrite the compiler executable here from gcc to msvc. If that works we can add a compiler-property to use either gcc or msvc depending on the OS.

baentsch commented 3 years ago

That's what I did (and meant to say with "After replacing gcc with cl"); hence we see cl used in the command above. Unfortunately -o didn't get replaced by -Fo as per the "advertisement"...

jimouris commented 3 years ago

I think the issue is that in the pom.xml file I have specified

<compilerProvider>generic-classic</compilerProvider>
<compilerExecutable>gcc</compilerExecutable>

Changing the second line (gcc -> msvc) doesn't work, but if you change the first line and omit the second like:

<compilerProvider>msvc</compilerProvider>

The plugin changed -o to /Fo:

/bin/sh -c cd '/home/jimouris/repos/liboqs-java' && 'cl.exe' '-I/usr/lib/jvm/java-1.8.0-openjdk-amd64/include' '-I/usr/lib/jvm/java-1.8.0-openjdk-amd64/include/linux' '-fPIC' '-Wall' '-O3' '-I/usr/local/include' '-I/home/jimouris/repos/liboqs-java/src/main/c' '-I/home/jimouris/repos/liboqs-java/target/generated-sources/annotations' '/Fo/home/jimouris/repos/liboqs-java/target/objs/handle.o' '-c' '/home/jimouris/repos/liboqs-java/src/main/c/handle.c'

(running the above on linux just to test the plugin options)

baentsch commented 3 years ago

This indeed creates proper object files -- but then the linking fails completely: It looks like the linker takes the JAR file as target instead of the shared library:

>mvn package -P windows -Dliboqs.include.dir="c:\Users\Administrator\git\liboqs\build\include" -Dliboqs.lib.dir="c:\Users\Administrator\git\liboqs\build\lib"
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------< org.openquantumsafe:liboqs-java >-------------------
[INFO] Building liboqs-java: Java wrapper for liboqs 1.0
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-antrun-plugin:3.0.0:run (default) @ liboqs-java ---
[INFO] Executing tasks
[INFO] Executed tasks
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:resources (default-resources) @ liboqs-java ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 2 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ liboqs-java ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- native-maven-plugin:1.0-alpha-9:initialize (generate-shared-oqs-library) @ liboqs-java ---
[INFO]
[INFO] --- native-maven-plugin:1.0-alpha-9:compile (generate-shared-oqs-library) @ liboqs-java ---
[INFO]
[INFO] --- native-maven-plugin:1.0-alpha-9:link (generate-shared-oqs-library) @ liboqs-java ---
[INFO] cmd.exe /X /C "link.exe /out:C:\Users\Administrator\git\liboqs-java\target\liboqs-java.jar target\objs\handle.obj target\objs\KEMs.obj target\objs\KeyEncapsulation.obj target\objs\Rand.obj target\objs\Signature.obj target\objs\Sigs.obj"
Microsoft (R) Incremental Linker Version 14.28.29910.0
Copyright (C) Microsoft Corporation.  All rights reserved.

   Creating library C:\Users\Administrator\git\liboqs-java\target\liboqs-java.lib and object C:\Users\Administrator\git\liboqs-java\target\liboqs-java.exp
LINK : fatal error LNK1561: entry point must be defined
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  7.891 s
[INFO] Finished at: 2021-03-15T13:41:03Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:native-maven-plugin:1.0-alpha-9:link (generate-shared-oqs-library) on project liboqs-java: Error executing command line. Exit code:1561 -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

To get there I changed the pom.xml as follows: 1) Added windows profile:

        <profile>
            <id>windows</id>
            <activation>
                <activeByDefault>true</activeByDefault>
                <property>
                    <name>platform</name>
                    <value>windows</value>
                </property>
            </activation>
            <properties>
                <lib_name>liboqs-jni.dll</lib_name>
                <liboqs.include.dir>/usr/local/include</liboqs.include.dir>
                <liboqs.lib.dir>/usr/local/lib</liboqs.lib.dir>
                <java.os.include>-I${JAVA_HOME}/include -I${JAVA_HOME}/include/win32</java.os.include>
            </properties>
        </profile>

2) set compiler & linker as

                    <compilerProvider>msvc</compilerProvider>
                    <compilerStartOptions>
                        <compilerStartOption>${java.os.include}</compilerStartOption>
                        <compilerStartOption>-I${liboqs.include.dir}</compilerStartOption>
                    </compilerStartOptions>
                    <linkerProvider>msvc</linkerProvider>
                    <linkerOutputDirectory>target</linkerOutputDirectory>

Maybe time to read the full maven documentation alongside the one for MSVC...

christianpaquin commented 3 years ago

Can you re-run Maven with -e and -X switch to get full stack trace and debug output, @baentsch? Also, the following properties in the windows profile look wrong:

 <liboqs.include.dir>/usr/local/include</liboqs.include.dir>
 <liboqs.lib.dir>/usr/local/lib</liboqs.lib.dir>

I don't know if they cause the linking errors you are seeing.

baentsch commented 3 years ago

Also, the following properties in the windows profile look wrong:

Completely agree. But I'm running maven like the below (overruling these settings, I'd think):

C:\Users\Administrator\git\liboqs-java>mvn package -e -X -P windows -Dliboqs.include.dir="c:\Users\Administrator\git\liboqs\build\include" -Dliboqs.lib.dir="c:\Users\Administrator\git\liboqs\build\lib" > mvn.log

Can you re-run Maven with -e and -X switch to get full stack trace and debug output:

Thanks for checking and for the suggestion. Shows clearly that the jar and not the dll is link target:

mvn.log

So clearly no linker problem, but a maven config issue: Any idea, @jimouris ? Also attached the complete pom.xml:

pom.xml.txt

jimouris commented 3 years ago

I made a few changes and pushed to a new windows branch the pom.xml. I think the problem is with the linkerStartOptions and linkerEndOptions but I'm not sure what are the correct flags for MSVC.

In your log,

[INFO] cmd.exe /X /C "link.exe /out:C:\Users\Administrator\git\liboqs-java\target\liboqs-java.jar target\objs\handle.obj target\objs\KEMs.obj target\objs\KeyEncapsulation.obj target\objs\Rand.obj target\objs\Signature.obj target\objs\Sigs.obj"

it doesn't seem to include ${liboqs.lib.dir} at all. I think the correct flag is /DLL (instead of -L) but I cannot check it.

Also, it adds the /out at the beginning, which means it ignores the -o we had in the linkerEndOptions.

In the windows branch in the pom.xml I added some template code, but for the things I was not sure about and I could not test, I left them in comments (lines 99, 100, 107, and 108) here.

shikida commented 1 year ago

Hi. I am running windows 11 here, but without MSVC. Do you guys think it's feasible to use some gcc-based compiler (mingw) or even consider providing the windows binary? If there's something I can do to help, pls let me know.

christianpaquin commented 1 year ago

Do you guys think it's feasible to use some gcc-based compiler (mingw) or even consider providing the windows binary? If there's something I can do to help, pls let me know.

Can you get it to work with mingw? If so, you can document how or explain what we would need to change.

johngray-dev commented 7 months ago

I finally got it working in windows but it did require a bunch of updates to the POM file. If you like I could submit a fix for this.

baentsch commented 7 months ago

I finally got it working in windows but it did require a bunch of updates to the POM file. If you like I could submit a fix for this.

That would be very welcome!

johngray-dev commented 7 months ago

Working POM file:

Directions:

Build liboqs on Windows as per regular instructions included for liboqs. I like using cmake-gui to make the windows solution files. You can get cmake (that comes with cmake-gui) here: https://cmake.org/download/ -a) Set cmake source code to the location where you cloned the main liboqs repository -b) Set where to build binary files wherever you want the output. This location is important for building with liboqs-java. -c) Make sure you add an entry called BUILD_SHARED_LIBS as a BOOL and checkmark it (this will ensure the liboqs.dll is built) -d) Generate the solution files, and build with VC 2019. The main liboqs.sln will be in the folder you specify in 1b above. -e) Open the liboqs.sln file in visual studio and compile the project. liboqs.dll will be in the output folder from 1b in a folder called bin (and probably under a Debug or Release folder).
To build Liboqs-java on Windows: -a) Open an "x64 Native Tools Command Prompt for VS 2019" (or whatever version of VS code you have). You can find it in the start menu folder for Visual Studio 2019.
-b) In the x64 Native Tools Command Prompt cd to the directory where liboqs-java was cloned -c) Issue the command "mvn package -P windows "-Dliboqs.include.dir=[path to liboqs include dir]" "-Dliboqs.lib.dir=[path to liboqs lib dir] where [path to liboqs include dir] is in the path from 1b inside the include folder.
[path to liboqs lib dir] is in the path from 1b inside the lib folder. -d) If all is well, the oqs-jni.dll and liboqs-java.jar should be in the liboqs-java projects target folder.

Here is a tested POM file (Test by two people at Entrust with Visual Studio 2019).

pom.zip

democuritus commented 3 weeks ago

SWilson4 commented 3 weeks ago

Hi @democuritus, I'm having trouble understanding what's going on based on the screenshot provided above. If you have a bug report, please open a new issue or discussion and provide more details about your setup and what you're trying to do.

democuritus commented 3 weeks ago

It seems completely impossible to integrate the liboq library into my JAVA project on Windows And the toolchain I used was the built-in compiler of vS 2022 throughout the entire process I liboqs.zip really hope you can point out my problem to me. I have already packaged and uploaded my compiled DLL and jar files

democuritus commented 3 weeks ago

thaks you very much for responding my question! Here is my process of compiling and packaging. Open Visual Studio "x64 Native Tools Command Prompt" , input cmake -G "Visual Studio 17 2022" -A x64 DBUILD_SHARED_LIBS=ON then cmake --build . --config Release to generate oqs.dll in liboqs-main\bin\Debug and Open Visual Studio "x64 Native Tools Command Prompt" into the liboqs-java project root directory, input jar cvf liboqs-java.jar -C target/classes then jar uvf liboqs-java.jar -C src/main/resources jar tf liboqs-java.jar to generate liboqs-jni.dll and liboqs-java.jar I don't know if the toolchains I use are compatible with each other

SWilson4 commented 3 weeks ago

I'm afraid I'm not very knowledgeable about either Windows or Java, but I'll try to help.

Two questions:

Which version of liboqs are you using?
Have you tried with the name "MAYO-3" instead of "mayo_3"?

democuritus commented 3 weeks ago

Moreover, under Windows, Maven and Mingw seem to encounter compilation errors when generating DLLs and JARs. This is the only method I have explored that can generate these target files.I use：

        <groupId>org.openquantumsafe</groupId>
        <artifactId>liboqs-java</artifactId>
        <version>1.0.0</version>
        <scope>system</scope>
        <systemPath>${project.basedir}/lib/liboqs-java.jar</systemPath>
    </dependency>//

to import these files into my springbootproject, then had written a test class,here is my source code:

package com.yzy.springboot.utils; import org.openquantumsafe.Signature; import java.util.Base64; public class QuantumResistantTest { public static void main(String[] args) { try { // load dll System.load("D:\liboqs-main\liboqs-main\bin\Debug\oqs.dll"); System.out.println("Library loaded successfully");

        // create Signature object
        Signature signature = new Signature("mayo_3");
        System.out.println("Signature object created successfully");

        // generate keypairs
        byte[] publicKey = signature.generate_keypair();
        byte[] privateKey = signature.export_secret_key();
        System.out.println("Key pair generated successfully");
        System.out.println("Public Key: " + Base64.getEncoder().encodeToString(publicKey));
        System.out.println("Private Key: " + Base64.getEncoder().encodeToString(privateKey));

        // sign the message
        String messageStr = "Hello, post-quantum world!";
        byte[] message = messageStr.getBytes();
        byte[] signatureBytes = signature.sign(message);
        System.out.println("Message signed successfully");
        System.out.println("Signature: " + Base64.getEncoder().encodeToString(signatureBytes));

        // verify sig
        boolean isValid = signature.verify(message, signatureBytes, publicKey);
        if (isValid) {
            System.out.println("Signature verification successful");
        } else {
            System.out.println("Signature verification failed");
        }

        // print sig info
        signature.print_details();

        // release resources
        signature.dispose_sig();
        System.out.println("Signature object disposed successfully");

    } catch (UnsatisfiedLinkError e) {
        System.err.println("Failed to load library: " + e.getMessage());
        e.printStackTrace();
    } catch (RuntimeException e) {
        System.err.println("Error: " + e.getMessage());
        e.printStackTrace();
    }
}

}

democuritus commented 3 weeks ago

Thank you very much for your reply, I am using liboqs C library version 0.11.0, and liboqs-java version 0.1.0

democuritus commented 3 weeks ago

I tried with the name "MAYO-3", and still doesn't work.

SWilson4 commented 3 weeks ago

I'm afraid that this goes beyond my knowledge of either Windows or Java. I did confirm (using Ubuntu 24 / OpenJDK 11) that liboqs-java can create a MAYO-3 signature (using the name "MAYO-3", not "mayo_3"—the latter leads to an exception). This project is also not officially supported for Windows.

I'd suggest you try the instructions and pom file above if you haven't already.

democuritus commented 3 weeks ago

thanks you for confriming, this information is important. that means my jar and oqs.dll have no problem? I will try newer java version.

---Original--- From: "Spencer @.> Date: Sat, Oct 19, 2024 03:32 AM To: @.>; Cc: @.**@.>; Subject: Re: [open-quantum-safe/liboqs-java] Enable build on Windows (#1)

I'm afraid that this goes beyond my knowledge of either Windows or Java. I did confirm (using Ubuntu 24 / OpenJDK 11) that liboqs-java can create a MAYO-3 signature (using the name "MAYO-3", not "mayo_3"—the latter leads to an exception).

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

democuritus commented 3 weeks ago

hi @johngray-dev I don't know whether my oqs.dll,oqs-jni.dll,oqs-java.jar is generated correctly. I use your method to get these packages ,but still got error screenshots； D:\liboqs-java-master>mvn clean package -P windows "-Dliboqs.include.dir=E:/liboqs-main/liboqs-main/build/include" "-Dliboqs.lib.dir=E:/liboqs-main/liboqs-main/build/lib/Release" [INFO] Scanning for projects... [INFO] [INFO] ------------------< org.openquantumsafe:liboqs-java >------------------- [INFO] Building liboqs-java: Java wrapper for liboqs 1.0 [INFO] from pom.xml [INFO] --------------------------------[ jar ]--------------------------------- [INFO] [INFO] --- clean:3.2.0:clean (default-clean) @ liboqs-java --- [INFO] Deleting D:\liboqs-java-master\target [INFO] [INFO] --- antrun:3.0.0:run (default) @ liboqs-java --- [INFO] Executing tasks [INFO] Executed tasks [INFO] [INFO] --- resources:3.1.0:resources (default-resources) @ liboqs-java --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 0 resource [INFO] [INFO] --- compiler:3.8.1:compile (default-compile) @ liboqs-java --- [INFO] Changes detected - recompiling the module! [INFO] Compiling 9 source files to D:\liboqs-java-master\target\classes [INFO] [INFO] --- native:1.0-alpha-9:initialize (generate-shared-oqs-library) @ liboqs-java --- [INFO] [INFO] --- native:1.0-alpha-9:compile (generate-shared-oqs-library) @ liboqs-java --- [INFO] cmd.exe /X /C "cl.exe -I"E:\java\jdk1.8.0_321/include" -I"E:\java\jdk1.8.0_321/include/win32" -IE:/liboqs-main/liboqs-main/build/include -ID:\liboqs-java-master\src\main\c -ID:\liboqs-java-master\target\generated-sources\annotations /FoD:\liboqs-java-master\target\objs\handle.obj -c D:\liboqs-java-master\src\main\c\handle.c" 用于 x64 的 Microsoft (R) C/C++ 优化编译器 19.41.34123 版版权所有(C) Microsoft Corporation。保留所有权利。

handle.c [INFO] cmd.exe /X /C "cl.exe -I"E:\java\jdk1.8.0_321/include" -I"E:\java\jdk1.8.0_321/include/win32" -IE:/liboqs-main/liboqs-main/build/include -ID:\liboqs-java-master\src\main\c -ID:\liboqs-java-master\target\generated-sources\annotations /FoD:\liboqs-java-master\target\objs\KEMs.obj -c D:\liboqs-java-master\src\main\c\KEMs.c" 用于 x64 的 Microsoft (R) C/C++ 优化编译器 19.41.34123 版版权所有(C) Microsoft Corporation。保留所有权利。

KEMs.c [INFO] cmd.exe /X /C "cl.exe -I"E:\java\jdk1.8.0_321/include" -I"E:\java\jdk1.8.0_321/include/win32" -IE:/liboqs-main/liboqs-main/build/include -ID:\liboqs-java-master\src\main\c -ID:\liboqs-java-master\target\generated-sources\annotations /FoD:\liboqs-java-master\target\objs\KeyEncapsulation.obj -c D:\liboqs-java-master\src\main\c\KeyEncapsulation.c" 用于 x64 的 Microsoft (R) C/C++ 优化编译器 19.41.34123 版版权所有(C) Microsoft Corporation。保留所有权利。

KeyEncapsulation.c [INFO] cmd.exe /X /C "cl.exe -I"E:\java\jdk1.8.0_321/include" -I"E:\java\jdk1.8.0_321/include/win32" -IE:/liboqs-main/liboqs-main/build/include -ID:\liboqs-java-master\src\main\c -ID:\liboqs-java-master\target\generated-sources\annotations /FoD:\liboqs-java-master\target\objs\Rand.obj -c D:\liboqs-java-master\src\main\c\Rand.c" 用于 x64 的 Microsoft (R) C/C++ 优化编译器 19.41.34123 版版权所有(C) Microsoft Corporation。保留所有权利。

Rand.c [INFO] cmd.exe /X /C "cl.exe -I"E:\java\jdk1.8.0_321/include" -I"E:\java\jdk1.8.0_321/include/win32" -IE:/liboqs-main/liboqs-main/build/include -ID:\liboqs-java-master\src\main\c -ID:\liboqs-java-master\target\generated-sources\annotations /FoD:\liboqs-java-master\target\objs\Signature.obj -c D:\liboqs-java-master\src\main\c\Signature.c" 用于 x64 的 Microsoft (R) C/C++ 优化编译器 19.41.34123 版版权所有(C) Microsoft Corporation。保留所有权利。

Signature.c [INFO] cmd.exe /X /C "cl.exe -I"E:\java\jdk1.8.0_321/include" -I"E:\java\jdk1.8.0_321/include/win32" -IE:/liboqs-main/liboqs-main/build/include -ID:\liboqs-java-master\src\main\c -ID:\liboqs-java-master\target\generated-sources\annotations /FoD:\liboqs-java-master\target\objs\Sigs.obj -c D:\liboqs-java-master\src\main\c\Sigs.c" 用于 x64 的 Microsoft (R) C/C++ 优化编译器 19.41.34123 版版权所有(C) Microsoft Corporation。保留所有权利。

Sigs.c [INFO] [INFO] --- native:1.0-alpha-9:link (generate-shared-oqs-library) @ liboqs-java --- [INFO] cmd.exe /X /C "link /DLL /LIBPATH:E:/liboqs-main/liboqs-main/build/lib/Release oqs.lib oqs-internal.lib Advapi32.lib /out:D:\liboqs-java-master\target\liboqs-java.jar target\objs\handle.obj target\objs\KEMs.obj target\objs\KeyEncapsulation.obj target\objs\Rand.obj target\objs\Signature.obj target\objs\Sigs.obj /out:D:\liboqs-java-master/target/oqs-jni.dll" Microsoft (R) Incremental Linker Version 14.41.34123.0 Copyright (C) Microsoft Corporation. All rights reserved.

正在创建库 D:\liboqs-java-master\target\oqs-jni.lib 和对象 D:\liboqs-java-master\target\oqs-jni.exp [INFO] [INFO] --- resources:3.1.0:copy-resources (copy-resources) @ liboqs-java --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 0 resource [INFO] [INFO] --- resources:3.1.0:testResources (default-testResources) @ liboqs-java --- [INFO] Not copying test resources [INFO] [INFO] --- compiler:3.8.1:testCompile (default-testCompile) @ liboqs-java --- [INFO] Not compiling test sources [INFO] [INFO] --- surefire:2.22.0:test (default-test) @ liboqs-java --- [INFO] Tests are skipped. [INFO] [INFO] --- jar:3.4.1:jar (default-jar) @ liboqs-java --- [INFO] Building jar: D:\liboqs-java-master\target\liboqs-java.jar [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 3.760 s [INFO] Finished at: 2024-10-19T20:06:37+08:00 [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-jar-plugin:3.4.1:jar (default-jar) on project liboqs-java: You have to use a classifier to attach supplemental artifacts to the project instead of replacing them. -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException and here is my generated files: liboqs.zip liboqs.zip could you please help me? I use the recommend method on Ubuntu 22.04 VMware (using JDK 11) and encountered many unknow errors

democuritus commented 3 weeks ago

I successed creating a MAYO-3 signature in windows! appriciate for @SWilson4 your help! really appriciate!!

JulioCesarRuizG commented 2 weeks ago

I am trying to deploy the project for my thesis, but it isn't working. I only need a JAR file to use the post-quantum algorithms.

AdijeShen commented 2 weeks ago

Working POM file:

Directions:

Build liboqs on Windows as per regular instructions included for liboqs. I like using cmake-gui to make the windows solution files. You can get cmake (that comes with cmake-gui) here: https://cmake.org/download/ -a) Set cmake source code to the location where you cloned the main liboqs repository -b) Set where to build binary files wherever you want the output. This location is important for building with liboqs-java. -c) Make sure you add an entry called BUILD_SHARED_LIBS as a BOOL and checkmark it (this will ensure the liboqs.dll is built) -d) Generate the solution files, and build with VC 2019. The main liboqs.sln will be in the folder you specify in 1b above. -e) Open the liboqs.sln file in visual studio and compile the project. liboqs.dll will be in the output folder from 1b in a folder called bin (and probably under a Debug or Release folder).

To build Liboqs-java on Windows: -a) Open an "x64 Native Tools Command Prompt for VS 2019" (or whatever version of VS code you have). You can find it in the start menu folder for Visual Studio 2019. -b) In the x64 Native Tools Command Prompt cd to the directory where liboqs-java was cloned -c) Issue the command "mvn package -P windows "-Dliboqs.include.dir=[path to liboqs include dir]" "-Dliboqs.lib.dir=[path to liboqs lib dir] where [path to liboqs include dir] is in the path from 1b inside the include folder. [path to liboqs lib dir] is in the path from 1b inside the lib folder. -d) If all is well, the oqs-jni.dll and liboqs-java.jar should be in the liboqs-java projects target folder.

Here is a tested POM file (Test by two people at Entrust with Visual Studio 2019).

pom.zip

This guide looks very helpful, especially since Windows builds are a common pain point. Would there be plans to merge this POM file and build instructions into the master branch? It could help many developers who face similar Windows build challenges.

democuritus commented 2 weeks ago

you can try the zip under my comment,and get according deplory to adopt your private computer.that dll and jar is tested working

---Original--- From: @.> Date: Wed, Oct 23, 2024 06:38 AM To: @.>; Cc: @.**@.>; Subject: Re: [open-quantum-safe/liboqs-java] Enable build on Windows (#1)

I am trying to deploy the project for my thesis, but it isn't working. I only need a JAR file to use the post-quantum algorithms.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

AdijeShen commented 1 week ago

I've found a solution to build liboqs-java on Windows and created a working demo project. Here are the details:

Modified build approach (https://github.com/AdijeShen/liboqs-java):
- Uses liboqs.a instead of liboqs.dll/liboqs.so, eliminating system directory installation
- Requires only compiling liboqs.a from the ./liboqs/ folder
- Works with MinGW-w64 GCC (11.5.0+), CMake, JDK 1.8, and Maven 3.8.8
Demo project (https://github.com/AdijeShen/oqs-java-kem-test):
- Shows how to use the built jar file
- Works on both Windows and Linux
- Simple example running KEM methods

Please note that while this solution works well across Linux/macOS/Windows, it may not be suitable for merging into the main branch since it uses static libraries (.a) instead of dynamic libraries. However, if anyone is interested in building and running liboqs-java on Windows, feel free to check out my repository for a working implementation.

baentsch commented 1 week ago

Thanks for your work and this report @AdijeShen !

it may not be suitable for merging into the main branch since it uses static libraries (.a) instead of dynamic libraries.

Why do you think so? Building liboqs statically is a valid and supported procedure. Thus, a PR to make things easy for others will surely be welcome.

AdijeShen commented 1 week ago

Thanks for your work and this report @AdijeShen !

it may not be suitable for merging into the main branch since it uses static libraries (.a) instead of dynamic libraries.

Why do you think so? Building liboqs statically is a valid and supported procedure. Thus, a PR to make things easy for others will surely be welcome.

Thank you for clarifying that liboqs static builds are supported - I'll prepare the necessary changes and documentation for PR.

open-quantum-safe / liboqs-java

Enable build on Windows #1