potatosalad / erlang-libdecaf

ed448goldilocks (libdecaf) NIF with timeslice reductions for Erlang and Elixir.
MIT License
29 stars 16 forks source link

FreeBSD 10.3R amd64 failure #2

Closed dch closed 5 years ago

dch commented 8 years ago

super interesting library! as title, I got this failure during goldilocks on FreeBSD - any ideas?

dch@wintermute /r/erlang-libdecaf> gmake
 DEPEND libdecaf.d
 ERLC   libdecaf.erl libdecaf_curve25519.erl libdecaf_curve448.erl libdecaf_sha2.erl libdecaf_sha3.erl
 APP    libdecaf.app.src
gmake -C /ramdisk/erlang-libdecaf/c_src
gmake[1]: Entering directory '/ramdisk/erlang-libdecaf/c_src'
Cloning into '/ramdisk/erlang-libdecaf/.ed448goldilocks'...
remote: Counting objects: 4180, done.
remote: Total 4180 (delta 0), reused 0 (delta 0), pack-reused 4180
Receiving objects: 100% (4180/4180), 1000.44 KiB | 843.00 KiB/s, done.
Resolving deltas: 100% (3114/3114), done.
Checking connectivity... done.
gmake[2]: Entering directory '/ramdisk/erlang-libdecaf/.ed448goldilocks'
python -B src/generator/template.py --per=curve --item=curve25519 --guard=decaf/ed255.h -o src/GENERATED/include/decaf/ed255.h src/per_curve/eddsa.tmpl.h
mkdir -p build/obj src/GENERATED/c build/obj \
    build/lib src/GENERATED/include build/bin build/obj/bin src/GENERATED/c src/GENERATED/include/decaf \
    build/obj/p25519 build/obj/curve25519 build/obj/p448 build/obj/ed448goldilocks src/GENERATED/c/decaf
touch build/obj/timestamp
cp -f src/public_include/decaf/sha512.h src/GENERATED/include/decaf/sha512.h
cp -f src/public_include/decaf/spongerng.h src/GENERATED/include/decaf/spongerng.h
cp -f src/public_include/decaf/common.h src/GENERATED/include/decaf/common.h
python -B src/generator/template.py --per=curve --item=ed448goldilocks --guard=decaf/point_448.h -o src/GENERATED/include/decaf/point_448.h src/per_curve/point.tmpl.h
python -B src/generator/template.py --per=curve --item=curve25519 --guard=decaf/point_255.h -o src/GENERATED/include/decaf/point_255.h src/per_curve/point.tmpl.h
python -B src/generator/template.py --per=curve --item=ed448goldilocks --guard=decaf/ed448.h -o src/GENERATED/include/decaf/ed448.h src/per_curve/eddsa.tmpl.h
cp -f src/public_include/decaf/shake.h src/GENERATED/include/decaf/shake.h
python -B src/generator/template.py --per=global --guard=decaf.h -o src/GENERATED/include/decaf.h src/public_include/decaf.tmpl.h
python -B src/generator/template.py --per=global --guard=decaf/crypto.h -o src/GENERATED/c/decaf/crypto.h src/include/decaf/crypto.tmpl.h
python -B src/generator/template.py --per=curve --item=ed448goldilocks --guard=src/GENERATED/c/decaf/crypto_448.h -o src/GENERATED/c/decaf/crypto_448.h src/per_curve/crypto.tmpl.h
python -B src/generator/template.py --per=curve --item=curve25519 --guard=src/GENERATED/c/decaf/crypto_255.h -o src/GENERATED/c/decaf/crypto_255.h src/per_curve/crypto.tmpl.h
python -B src/generator/template.py --per=field --guard=p448/`basename src/GENERATED/c/p448/f_field.h` --item=p448 -o src/GENERATED/c/p448/f_field.h src/per_field/f_field.tmpl.h
python -B src/generator/template.py --per=field --guard=p25519/`basename src/GENERATED/c/p25519/f_field.h` --item=p25519 -o src/GENERATED/c/p25519/f_field.h src/per_field/f_field.tmpl.h
cp -f src/public_include/decaf/spongerng.hxx src/GENERATED/include/decaf/spongerng.hxx
python -B src/generator/template.py --per=global --guard=decaf/eddsa.hxx -o src/GENERATED/include/decaf/eddsa.hxx src/public_include/decaf/eddsa.tmpl.hxx
cp -f src/public_include/decaf/sha512.hxx src/GENERATED/include/decaf/sha512.hxx
cp -f src/public_include/decaf/shake.hxx src/GENERATED/include/decaf/shake.hxx
cp -f src/public_include/decaf/secure_buffer.hxx src/GENERATED/include/decaf/secure_buffer.hxx
python -B src/generator/template.py --per=global --guard=decaf.hxx -o src/GENERATED/include/decaf.hxx src/public_include/decaf.tmpl.hxx
python -B src/generator/template.py --per=global --guard=decaf/crypto.hxx -o src/GENERATED/c/decaf/crypto.hxx src/include/decaf/crypto.tmpl.hxx
python -B src/generator/template.py --per=curve --item=curve25519 --guard=decaf/point_255.hxx -o src/GENERATED/include/decaf/point_255.hxx src/per_curve/point.tmpl.hxx
python -B src/generator/template.py --per=curve --item=curve25519 --guard=src/GENERATED/c/decaf/crypto_255.hxx -o src/GENERATED/c/decaf/crypto_255.hxx src/per_curve/crypto.tmpl.hxx
python -B src/generator/template.py --per=curve --item=curve25519 --guard=decaf/ed255.hxx -o src/GENERATED/include/decaf/ed255.hxx src/per_curve/eddsa.tmpl.hxx
python -B src/generator/template.py --per=curve --item=ed448goldilocks --guard=decaf/point_448.hxx -o src/GENERATED/include/decaf/point_448.hxx src/per_curve/point.tmpl.hxx
python -B src/generator/template.py --per=curve --item=ed448goldilocks --guard=src/GENERATED/c/decaf/crypto_448.hxx -o src/GENERATED/c/decaf/crypto_448.hxx src/per_curve/crypto.tmpl.hxx
python -B src/generator/template.py --per=curve --item=ed448goldilocks --guard=decaf/ed448.hxx -o src/GENERATED/include/decaf/ed448.hxx src/per_curve/eddsa.tmpl.hxx
cc -std=c99 -fno-strict-aliasing -pedantic -Wall -Wextra -Werror -Wunreachable-code -Wmissing-declarations -Wunused-function -Wno-overlength-strings  -Isrc/include -Isrc/GENERATED/include -Isrc/GENERATED/c -O2 -maes -mavx2 -mbmi2   -ffunction-sections -fdata-sections -fvisibility=hidden -fomit-frame-pointer -fPIC  -c -o build/obj/utils.o src/utils.c
cc -std=c99 -fno-strict-aliasing -pedantic -Wall -Wextra -Werror -Wunreachable-code -Wmissing-declarations -Wunused-function -Wno-overlength-strings  -Isrc/include -Isrc/GENERATED/include -Isrc/GENERATED/c -O2 -maes -mavx2 -mbmi2   -ffunction-sections -fdata-sections -fvisibility=hidden -fomit-frame-pointer -fPIC  -c -o build/obj/shake.o src/shake.c
In file included from src/shake.c:20:
src/include/portable_endian.h:16:10: error: 'le64toh' macro redefined [-Werror]
#       define le64toh(x) letoh64(x)
               ^
/usr/include/sys/endian.h:80:9: note: previous definition is here
#define le64toh(x)      ((uint64_t)(x))
        ^
src/shake.c:63:33: error: implicit declaration of function 'letoh64' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
    for (i=0; i<25; i++) a[i] = le64toh(a[i]);
                                ^
src/include/portable_endian.h:16:21: note: expanded from macro 'le64toh'
#       define le64toh(x) letoh64(x)
                          ^
2 errors generated.
Makefile:259: recipe for target 'build/obj/shake.o' failed
gmake[2]: *** [build/obj/shake.o] Error 1
gmake[2]: Leaving directory '/ramdisk/erlang-libdecaf/.ed448goldilocks'
Makefile:182: recipe for target '/ramdisk/erlang-libdecaf/.ed448goldilocks/build/lib/libdecaf.so.1' failed
gmake[1]: *** [/ramdisk/erlang-libdecaf/.ed448goldilocks/build/lib/libdecaf.so.1] Error 2
gmake[1]: Leaving directory '/ramdisk/erlang-libdecaf/c_src'
erlang.mk:1759: recipe for target 'app-c_src' failed
gmake: *** [app-c_src] Error 2
potatosalad commented 8 years ago

@dch Hmm...good question. This might be an upstream issue for ed448goldilocks, but I'll see if I can get it to build first on my FreeBSD box later today.

potatosalad commented 8 years ago

@dch I just released version 0.0.3 of libdecaf which should (hopefully) build properly for you. I've also submitted the fix upstream.

Also, beware the following two issues you might run into:

  1. The builtin version of clang (3.4.1) for FreeBSD 10.3 is too old to build the library correctly.
  2. The default ARCHFLAGS may not be correct for your system (defaults are -maes -mavx2 -mbmi2).

To remedy the first issue, installing clang/llvm 3.7 or 3.8 and forcing its use should be enough to get it to work. For example:

gmake CC=clang38 CXX=clang++38 CPP=clang-cpp38

For the second issue, you can disable the extra flags by adding the following to the end of the gmake command:

ARCHFLAGS=-Wgcc-compat

Alternatively, if you have the cpuflags command installed, you can specify something like this:

ARCHFLAGS="-Wgcc-compat `cpuflags`"

The full command might look like this:

gmake CC=clang38 CXX=clang++38 CPP=clang-cpp38 ARCHFLAGS="-Wgcc-compat `cpuflags`"

The Erlang tests can be run with gmake tests.

The C/C++ tests and benchmarks can be run by using cd .ed448goldilocks and runing gmake bench. Below are the results from my system:

Macro-benchmarks for Iso-Ed25519:
CFRG crypto benchmarks:
RFC 7748 keygen:           56.43 µs     17.72 k/s    146.70 kcy
RFC 7748 shared secret:   153.10 µs      6.53 k/s    398.00 kcy
EdDSA keygen:              55.70 µs     17.95 k/s    144.71 kcy
EdDSA sign:                62.37 µs     16.03 k/s    162.18 kcy
EdDSA verify:             172.51 µs      5.80 k/s    448.55 kcy

Toy crypto benchmarks:
Create private key:        63.16 µs     15.83 k/s    164.09 kcy
Sign:                      64.78 µs     15.44 k/s    168.41 kcy
Verify:                   199.12 µs      5.02 k/s    517.72 kcy
SharedSecret:             185.12 µs      5.40 k/s    483.19 kcy

Toy protocol benchmarks:
Spake2ee c+s:             575.02 µs      1.74 k/s      1.50 Mcy
Spake2ee c+s aug:         945.16 µs      1.06 k/s      2.46 Mcy
FHMQV c+s:                641.21 µs      1.56 k/s      1.67 Mcy
TripleDH anon c+s:          1.15 ms    871.43  /s      2.98 Mcy

Macro-benchmarks for Ed448-Goldilocks:
CFRG crypto benchmarks:
RFC 7748 keygen:          145.49 µs      6.87 k/s    378.32 kcy
RFC 7748 shared secret:   487.74 µs      2.05 k/s      1.27 Mcy
EdDSA keygen:             147.47 µs      6.78 k/s    383.46 kcy
EdDSA sign:               158.41 µs      6.31 k/s    411.85 kcy
EdDSA verify:             382.42 µs      2.61 k/s    994.22 kcy

Toy crypto benchmarks:
Create private key:       101.33 µs      9.87 k/s    263.41 kcy
Sign:                     158.52 µs      6.31 k/s    412.15 kcy
Verify:                   356.83 µs      2.80 k/s    927.67 kcy
SharedSecret:             271.99 µs      3.68 k/s    707.14 kcy

Toy protocol benchmarks:
Spake2ee c+s:             718.42 µs      1.39 k/s      1.87 Mcy
Spake2ee c+s aug:           1.24 ms    805.35  /s      3.23 Mcy
FHMQV c+s:                799.62 µs      1.25 k/s      2.08 Mcy
TripleDH anon c+s:          1.60 ms    626.68  /s      4.15 Mcy

Cycle calibration:          2.60 GHz

Let me know if you hit any other issues or have any questions.

dch commented 5 years ago

@potatosalad sorry for the 2 year delay in getting back to you :-( LGTM, compiles beautifully now on 12.0-RC3 and tests pass too! thanks so much for this.