search

open-quantum-safe / liboqs

C library for prototyping and experimenting with quantum-resistant cryptography
https://openquantumsafe.org/
Other
1.78k stars 436 forks source link

Collecting per-build environment algorithm failures #1000

Closed baentsch closed 3 years ago

baentsch commented 3 years ago

In support of #892 this is to document build failures by algorithm and build environment.

Using our CI tests, downstream integrations and manual tests supporting the wiki build instructions, all algorithms not listed below should be assumed to build and work OK on

Failures: BIKE: fails to build completely under MSVC, thousands of errors, e.g., grafik

BIKE: fails also under cygwin (less dramatic), e.g.

[22/1510] Building ASM object src/kem/.../bike1_l1_fo.dir/additional/gf_mul.S.o
FAILED: src/kem/bike/CMakeFiles/bike1_l1_fo.dir/additional/gf_mul.S.o
/usr/bin/cc -DAVX2 -DFUNC_PREFIX=OQS_KEM_bike1_l1_fo -DLEVEL=1 -Iinclude -I/usr/include -fPIC -march=native -Werror -Wall -Wextra -Wpedantic -Wstrict-prototypes -Wshadow -Wformat=2 -Wfloat-equal -Wwrite-strings -O3 -fomit-frame-pointer -fdata-sections -ffunction-sections -Wl,--gc-sections -Wno-maybe-uninitialized -mavx2 -include /home/Administrator/git/liboqs/src/kem/bike/functions_renaming.h -MD -MT src/kem/bike/CMakeFiles/bike1_l1_fo.dir/additional/gf_mul.S.o -MF src/kem/bike/CMakeFiles/bike1_l1_fo.dir/additional/gf_mul.S.o.d -o src/kem/bike/CMakeFiles/bike1_l1_fo.dir/additional/gf_mul.S.o -c ../src/kem/bike/additional/gf_mul.S
../src/kem/bike/additional/gf_mul.S: Assembler messages:
../src/kem/bike/additional/gf_mul.S:70: Error: unknown pseudo-op: `.hidden'
../src/kem/bike/additional/gf_mul.S:71: Warning: .type pseudo-op used outside of .def/.endef: ignored.
../src/kem/bike/additional/gf_mul.S:71: Error: junk at end of line, first unrecognized character is `O'
../src/kem/bike/additional/gf_mul.S:130: Warning: .size pseudo-op used outside of .def/.endef: ignored.
../src/kem/bike/additional/gf_mul.S:130: Error: junk at end of line, first unrecognized character is `O'
../src/kem/bike/additional/gf_mul.S:187: Error: unknown pseudo-op: `.hidden'
../src/kem/bike/additional/gf_mul.S:188: Warning: .type pseudo-op used outside of .def/.endef: ignored.
../src/kem/bike/additional/gf_mul.S:188: Error: junk at end of line, first unrecognized character is `O'
../src/kem/bike/additional/gf_mul.S:232: Warning: .size pseudo-op used outside of .def/.endef: ignored.
../src/kem/bike/additional/gf_mul.S:232: Error: junk at end of line, first unrecognized character is `O'
../src/kem/bike/additional/gf_mul.S:283: Error: unknown pseudo-op: `.hidden'
../src/kem/bike/additional/gf_mul.S:284: Warning: .type pseudo-op used outside of .def/.endef: ignored.
../src/kem/bike/additional/gf_mul.S:284: Error: junk at end of line, first unrecognized character is `O'
../src/kem/bike/additional/gf_mul.S:315: Warning: .size pseudo-op used outside of .def/.endef: ignored.
../src/kem/bike/additional/gf_mul.S:315: Error: junk at end of line, first unrecognized character is `O'

Picnic: Fails under cygwin; probably unintended overload of system defines, e.g.:

/usr/bin/cc -DOQS -DPICNIC_STATIC -DWITH_KKW -DWITH_LOWMC_128_128_20 -DWITH_LOWMC_129_129_4 -DWITH_LOWMC_192_192_30 -DWITH_LOWMC_192_192_4 -DWITH_LOWMC_255_255_4 -DWITH_LOWMC_256_256_38 -DWITH_OPT -DWITH_SSE2 -DWITH_UNRUH -DWITH_ZKBPP -Iinclude -I../src/sig/picnic/external -fvisibility=hidden -march=native -Werror -Wall -Wextra -Wpedantic -Wstrict-prototypes -Wshadow -Wformat=2 -Wfloat-equal -Wwrite-strings -O3 -fomit-frame-pointer -fdata-sections -ffunction-sections -Wl,--gc-sections -Wno-maybe-uninitialized -Wno-missing-braces -std=gnu11 -MD -MT src/sig/picnic/CMakeFiles/picnic.dir/external/io.c.o -MF src/sig/picnic/CMakeFiles/picnic.dir/external/io.c.o.d -o src/sig/picnic/CMakeFiles/picnic.dir/external/io.c.o -c ../src/sig/picnic/external/io.c
In file included from ../src/sig/picnic/external/compat.h:62,
                 from ../src/sig/picnic/external/io.c:17:
../src/sig/picnic/external/endian_compat.h:151: error: "htobe16" redefined [-Werror]
  151 | #define htobe16(x) bswap16((x))
      |
In file included from /usr/include/machine/types.h:19,
                 from /usr/include/sys/types.h:224,
                 from /usr/include/stdio.h:61,
                 from ../src/sig/picnic/external/io.h:14,
                 from ../src/sig/picnic/external/io.c:14:
/usr/include/endian.h:42: note: this is the location of the previous definition
   42 | #define htobe16(x) __bswap_16(x)
      |
In file included from ../src/sig/picnic/external/compat.h:62,
                 from ../src/sig/picnic/external/io.c:17:
../src/sig/picnic/external/endian_compat.h:152: error: "htole16" redefined [-Werror]
  152 | #define htole16(x) ((uint16_t)(x))
      |
In file included from /usr/include/machine/types.h:19,
                 from /usr/include/sys/types.h:224,
                 from /usr/include/stdio.h:61,
                 from ../src/sig/picnic/external/io.h:14,
                 from ../src/sig/picnic/external/io.c:14:
/usr/include/endian.h:50: note: this is the location of the previous definition
   50 | #define htole16(x) (x)
      |
In file included from ../src/sig/picnic/external/compat.h:62,
                 from ../src/sig/picnic/external/io.c:17:
../src/sig/picnic/external/endian_compat.h:153: error: "be16toh" redefined [-Werror]
  153 | #define be16toh(x) bswap16((x))
      |
In file included from /usr/include/machine/types.h:19,
                 from /usr/include/sys/types.h:224,
                 from /usr/include/stdio.h:61,
                 from ../src/sig/picnic/external/io.h:14,
                 from ../src/sig/picnic/external/io.c:14:
/usr/include/endian.h:46: note: this is the location of the previous definition
   46 | #define be16toh(x) __bswap_16(x)
      |
In file included from ../src/sig/picnic/external/compat.h:62,
                 from ../src/sig/picnic/external/io.c:17:
../src/sig/picnic/external/endian_compat.h:154: error: "le16toh" redefined [-Werror]
  154 | #define le16toh(x) ((uint16_t)(x))

Question now: How to proceed? Widen the matrix/check more environments (gcc 9, 10 on Linux; M1 on OSX; msys2/mingw in Win)? Fix these (e.g., could #887 make BIKE be usable on Windows)? "Just" document in #892? What else?

Feedback solicited @jschanck @dstebila @christianpaquin @xvzcf

jschanck commented 3 years ago

The AVX2 implementation of BIKE includes some .S files that are ELF-only. These are incompatible with macOS and Windows. The CMakeLists file already handles macOS, so we just need to do the same for Windows here: https://github.com/open-quantum-safe/liboqs/blob/530d7059b27946db8941a8c8d012740a8d03f9c2/src/kem/bike/CMakeLists.txt#L19 That should fix Cygwin builds. But MSVC requires some (minor) changes to the code. One issue is the use of __attribute__((aligned(n))). I don't have a Windows setup to do the development on. And I'm not sure it's worth it given we are waiting on a new BIKE implementation.

For Picnic in Cygwin we need to add the following to the list of config tests on lines 44-77 of endian_compat.h

#if defined(__CYGWIN__)
#include <endian.h>
#define HAVE_HOSTSWAP
#endif
baentsch commented 3 years ago

Thanks very much @jschanck : Just finished validating your suggestions above on a Windows AWS VM (access credentials via OQS AWS account if you want to use it too). Implemented in PR #1001. With this BIKE is now working on Windows & Picnic with cygwin; waiting for #887 for an MSVC fix for BIKE.

dstebila commented 3 years ago

Is this still required? Do our new algorithm datasheets adequately document this?

baentsch commented 3 years ago

887 should already have closed this. The datasheets further document things well.