Issue during Make of PQS OpenSSL

[CobbleVision]

Dear PQS Team,

I am running into the following problem after a successful build of the corresponding library. I am not sure how to further debug the error, as searching for the gcc commands using grep did not find the correct line. What could I do to solve the issue and install the library?

${LDCMD:-gcc} -pthread -m64 -Ioqs/include -Wa,--noexecstack -Wall -O3 -L. -Loqs/lib  \
    -o test/buildtest_c_srp test/buildtest_srp.o \
     -lssl -lcrypto -ldl -pthread -loqs -lm
rm -f test/buildtest_c_txt_db
/usr/bin/ld: cannot find -loqs
collect2: error: ld returned 1 exit status

Is it an issue with the capital, as using -Loqs does not seem to throw any errors! Perhaps you'll find the problem quicker, as if I spend more time trying to identify the problem. Should oqs be placed as loqs?

Best regards, Cob

[baentsch]

The question is where in your folder structure(s) you issue the command above? For the above to work, you must execute this in a folder that has a subfolder with the oqs libraries installed (as per these instructions ).

[CobbleVision]

Dear Baentsch,

I believe that I went through the process as described on the github page. Here is a detailed description of what I did. I am on Ubuntu 18.04 with Kernel 5.0.0-23-generic.

Please Note #1: Today the output changed from showing an error from the previous description to not showing any error/success at all. Please Note #2: As i completed the process before some commands returned "nothing to do".

Step by Step:

1) Cloned the OpenSSL PQS Repo 2) Cloned the liboqs Repo into the OpenSSL PQS Repo using Git and renamed the folder to loqs 3) Used sudo autoreconf -i in the loqs folder -> No output/No Error 4) Used ./configure --prefix=<PathToFolder>/OpenSSL_PQS/oqs --enable-shared=no -> Output Attached as file. 04_configure_loqs_output.txt 5) Used sudo make -j -> Looks fine -> Output attached as file 05_make_j_output.txt 6) Used sudo make install -> Looks fine -> Output attached as file 06_make_install_output.txt 7) Used sudo libtool --finish /usr/local/lib -> Looks fine 07_libtool_output.txt

8) Used sudo ./Configure no-shared linux-x86_64 -lm in OpenSSL PQS Folder -> Looks Fine -> Output attached as file 08_OpenSSL_PQM_loqs_output.txt 9) Used sudo make -j -> No Error, although the end looks as if it ended in the middle of the make process -> Output Attached 09_OpenSSL_PQM_Make_Output.txt

10) Verified OpenSSL -> sudo openssl genpkey -algorithm oqs_kem_default -out PQS_srv.key -> Algorithm not found

[baentsch]

This all looks good (except renaming liboqs in step 2 into the openssl git branch is not required: The make install deploys the liboqs libraries into the right location when passing the right location in step 4).

However, you seem to run (the system) openssl in step 10 without having installed (the new version) first (make install in OpenSSL missing as step 9a). That said, please be aware that make install isn't working completely correct right now (please see #128 for details).

In general, what about first trying out things without installing oqs-openssl over your system openssl, say using something along the lines of ./apps/openssl <your command here>? Operation as root/sudo then also shouldn't be necessary for such a "test-drive".

[CobbleVision]

Dear Baentsch,

I tested the make install command on my device. There was no error messages, but also no success message. I cannot replicate #128 anymore, although I had the same problem before. However, so far I cannot use a new cipher/algorithm.

Using apps/openssl genpkey -algorithm oqs_kem_default -out PQS_srv.key from the custom folder results in "algorithm not found".

Testing just the apps/openssl returns only the regular cyphers and not the quantum ones. I suppose the libqos build process is failing or not working as expected.

I might try to move some file locations today to see if that changes anything.

Best, Cob

[baentsch]

Dear Cob, this is very weird. What does ./openssl list -public-key-algorithms return (when executed in apps, of course :)? Not a single QSC alg??

In addition, did you do this ? Sorry, I've never used the key_default mechanism but always wanted to specify specific algorithms at runtime.

[CobbleVision]

Dear Baentsch,

1) I moved some files and tried to play with the oqs and lib folders, but did not get any further yet. The openssl library seems to updated and built, as the last change matched the last make.

2) I tried setting the default algorithm and rebuilt, but genpkey does not seem to offer the option of using a default algorithm. So far it is the same result.

3) I am using sudo apps/openssl enc --ciphers, which results in a list of the default openSSL algorithms. Using the original openssl libraries returns the same results.

Perhaps sudo could be causing issues? However, not using sudo fails, as access to /urs/lib requires sudo access.

Best regards, Cob

[baentsch]

Dear Cob, for me building liboqs and openssl (without installing openssl) absolutely works without sudo/root. Why/where do need write access to /usr/lib? Does the list pk-algs (see previous question) show any QSC algorithms? Greetings, --Michael

[CobbleVision]

Dear Baentsch,

using the previous command ./openssl list -public-key-algorithms results in the same list of algorithms, without any PQS algorithms.

Write access is required for make of the OpenSSL Build. This is the failure code:

*** Installing runtime libraries
*** Installing development files
install ./include/openssl/aes.h -> /usr/local/include/openssl/aes.h
cp: cannot create regular file '/usr/local/include/openssl/aes.h': Permission denied
Makefile:321: recipe for target 'install_dev' failed
make: *** [install_dev] Error 1

Best, Cob

[baentsch]

Hmm, this indeed indicates make install has been executed for openssl (which is risky without setting --prefix during openssl Configure/your step 8 as that could mess with your system openssl..). I just checked your log files again and they show indeed a correctly built and linking openssl. I can't understand how/why it shouldn't list the QSC algs... Can you run nm on your apps/openssl executable to see whether QSC symbols are in there (e.g., OQS_SIG_dilithium_2_new)?

Otherwise, I'd recommend a clean-slate start, doing everything without sudo up to and excluding make install in the OpenSSL build folder. What works fine for me is cloning and building liboqs and openssl in two separate folders next to each other.

Last alternative, if you're into Docker, you might want to give an "experimental" feature a try and run docker run -it openqsafe/liboqs-ubuntu-bionic-dev which should give you a root shell into a Docker image with a 'conservatively' (i.e., non-'/usr') built-and-installed OQS-openssl (and all required libraries for building further apps on top).

[CobbleVision]

Thanks for the support so far. Using nm does not show any QSC symbols / algs. I checked manually as well as with grep.

I'll retry to build the library in the coming days on a different system. Since I am living in an area with slow internet speed, I'll try the docker variant as soon as I have a better connection.

It might also be the hardware causing issues!

Best, Cob

[baentsch]

@CobbleVision , any news on the above? We switched the build system rather drastically, so you may want to re-try. If we don't hear back, we'll close this issue sometime next week.

[CobbleVision]

Thanks for the update. I haven't come back to the library yet. Currently busy with some higher priority infrastructure topics. I'll try it once I am back on top of my systems.

[baentsch]

OK - thanks for the tentative feedback. We'll then close. Please create a new issue should you still have (if at all, most probably new/different) problems with the new build environment.