Closed alexandervdm closed 3 years ago
Does it mean that ../mcl/src/dump_code | head
does not produce any output?
ex.
bls-eth-go-binary% ../mcl/src/dump_code | head
align 16
dq 0xffffffff00000001,0x53bda402fffe5bfe,0x3339d80809a1d805,0x73eda753299d7d48,
segment .text
global mclx_Fr_addPre
align 16
mclx_Fr_addPre:
db 0x48,0x8b,0x06,0x48,0x03,0x02,0x48,0x89,0x07,0x48,0x8b,0x46,0x08,0x48,0x13,0x42,
db 0x08,0x48,0x89,0x47,0x08,0x48,0x8b,0x46,0x10,0x48,0x13,0x42,0x10,0x48,0x89,0x47,
db 0x10,0x48,0x8b,0x46,0x18,0x48,0x13,0x42,0x18,0x48,0x89,0x47,0x18,0xc3,
segment .text
Correct. The executable runs successfully so there is no stacktrace or anything, but no output is produced.
Could you test the following command?
// create a.cpp
#include <stdio.h>
#include <cybozu/inttype.hpp>
int main()
{
printf("CYBOZU_HOST=%d\n", CYBOZU_HOST);
}
clang++ a.cpp -I ./include && ./a.out
If it puts CYBOZU_HOST=0
, then I should add macro for FreeBSD at https://github.com/herumi/mcl/blob/master/include/cybozu/inttype.hpp#L132
proofofbsd$ pwd
/home/alexander/Desktop/herumi-work/mcl
proofofbsd$ git rev-parse HEAD
bd5a3686924d4ef38f994bb400f87a684ee65fe8
proofofbsd$ clang++ a.cpp -I ./include && ./a.out
CYBOZU_HOST=1
proofofbsd$ uname -a
OpenBSD proofofbsd.workoffice.lan 6.8 GENERIC.MP#98 amd64
proofofbsd$ gmake CXX=clang++ obj/static_code.o
clang++ -o src/dump_code src/dump_code.cpp src/fp.cpp -g -I include -DMCL_DUMP_JIT -DMCL_MAX_BIT_SIZE=384 -DMCL_DONT_USE_OPENSSL -DMCL_USE_VINT -DMCL_SIZEOF_UNIT=8 -DMCL_VINT_FIXED_BUFFER
src/dump_code > src/static_code.asm
nasm -felf64 -o obj/static_code.o src/static_code.asm
proofofbsd$ ./src/dump_code
proofofbsd$
I've installed FreeBSD and test https://github.com/herumi/bls-eth-go-binary/commit/f7dd8401dd5774f9b634198c9eaf45465ad84d0e it runs well. If it does not run, then I'll install OpenBSD.
Sorry for the late reply. I hope I understood you correctly to try to apply the principle of patch f7dd8401dd5774f9b634198c9eaf45465ad84d0e to OpenBSD. Done as follows:
diff --git a/bls/bls.go b/bls/bls.go
index 273724e..b08ad55 100644
--- a/bls/bls.go
+++ b/bls/bls.go
@@ -14,7 +14,7 @@ package bls
#cgo darwin,amd64 LDFLAGS:-L${SRCDIR}/lib/darwin/amd64
#cgo darwin,arm64 LDFLAGS:-L${SRCDIR}/lib/darwin/arm64
#cgo windows,amd64 LDFLAGS:-L${SRCDIR}/lib/windows/amd64
-#cgo openbsd,amd64 LDFLAGS:-L${SRCDIR}/lib/openbsd/amd64
+#cgo openbsd,amd64 LDFLAGS:-L${SRCDIR}/lib/linux/amd64
#cgo freebsd,amd64 LDFLAGS:-L${SRCDIR}/lib/linux/amd64
typedef unsigned int (*ReadRandFunc)(void *, void *, unsigned int);
This unfortunately does not result in a successful build (using mcl-1.26 and bls-1.11 as the README currently suggests):
[.. omitted ..]
gmake -C ../mcl obj/static_code.o
gmake[1]: Entering directory '/home/alexander/Desktop/herumi-bugtry/mcl'
clang++ -o src/dump_code src/dump_code.cpp src/fp.cpp -g -I include -DMCL_DUMP_JIT -DMCL_MAX_BIT_SIZE=384 -DMCL_DONT_USE_OPENSSL -DMCL_USE_VINT -DMCL_SIZEOF_UNIT=8 -DMCL_VINT_FIXED_BUFFER
src/dump_code > src/static_code.asm
Illegal instruction (core dumped)
gmake[1]: *** [Makefile:250: src/static_code.asm] Error 132
gmake[1]: Leaving directory '/home/alexander/Desktop/herumi-bugtry/mcl'
gmake: *** [Makefile:59: ../mcl/obj/static_code.o] Error 2
I am revisiting this issue to hopefully upgrade to newer versions of your software, and I got some information that might be helpful. I prepared a fresh OpenBSD installation on the computer with the most modern CPU I own (i7-6700) to rule out any hardware issues and compilation of your software using the described steps was successfully without any issues.
I then tried the same on two AMD-based computers (FX-8320E & Opteron 6200) that unfortunately fail but for a different reason than what was mentioned above.
alexander@proofofobsd> ./src/dump_code
**** lines omitted *****
segment .text
global mclx_Fr_sqr
align 16
mclx_Fr_sqr:
db 0x48,0x89,0xf2,0xe9,0x88,0xfd,0xff,0xff,
segment .text
global mclx_Fr_preInv
align 16
mclx_Fr_preInv:
db 0x53,0x55,0x41,0x54,0x41,0x55,0x41,0x56,0x41,0x57,0x48,0x83,0xec,0x48,0x48,0x89,
db 0x7c,0x24,0x18,0x48,0x89,0xf0,0x4c,0x8b,0x30,0x4c,0x8b,0x78,0x08,0x48,0x8b,0x48,
db 0x10,0x48,0x89,0x0c,0x24,0x48,0x8b,0x48,0x18,0x48,0x89,0x4c,0x24,0x08,0x48,0x8d,
db 0x05,0x8b,0xf9,0xff,0xff,0x48,0x8b,0x10,0x48,0x8b,0x70,0x08,0x48,0x8b,0x78,0x10,
**** lines omitted *****
db 0x5f,0x41,0x5e,0x41,0x5d,0x41,0x5c,0x5d,0x5b,0xc3,
Illegal instruction (core dumped)
Running your test_util tool from xbyak/sample reveals the following on these CPUs:
64bit
vendor amd
mmx mmx2 cmov sse sse2 sse3 ssse3 sse41 sse42 popcnt aesni rdtscp osxsave(xgetvb) pclmulqdq avx lzcnt prefetchw
terminating with uncaught exception of type Xbyak::Error: can't protect
64bit
vendor amd
mmx mmx2 cmov sse sse2 sse3 ssse3 sse41 sse42 popcnt aesni rdtscp osxsave(xgetvb) pclmulqdq avx fma bmi1 lzcnt prefetchw f16c
terminating with uncaught exception of type Xbyak::Error: can't protect
While the POPCNT instruction seems to be available, both CPUs fail on the test function:
(gdb) bt
#0 thrkill () at /tmp/-:3
#1 0x0000034a67dc9cbe in _libc_abort () at /usr/src/lib/libc/stdlib/abort.c:51
#2 0x0000034aa7ee842c in abort_message (format=<optimized out>) at /usr/src/lib/libcxxabi/src/abort_message.cpp:77
#3 0x0000034aa7f0953a in demangling_terminate_handler () at /usr/src/lib/libcxxabi/src/cxa_default_handlers.cpp:62
#4 0x0000034aa7ee7eff in std::__terminate (func=0x0) at /usr/src/lib/libcxxabi/src/cxa_handlers.cpp:60
#5 0x0000034aa7f03a7c in __cxxabiv1::failed_throw (exception_header=0x34abab94500)
at /usr/src/lib/libcxxabi/src/cxa_exception.cpp:153
#6 0x0000034aa7f039db in __cxa_throw (thrown_object=0x34abab94580, tinfo=0x3482b621ba0 <typeinfo for Xbyak::Error>,
dest=<optimized out>) at /usr/src/lib/libcxxabi/src/cxa_exception.cpp:285
#7 0x000003482b609213 in Xbyak::CodeArray::CodeArray(unsigned long, void*, Xbyak::Allocator*) ()
#8 0x000003482b6067a1 in Xbyak::CodeGenerator::CodeGenerator(unsigned long, void*, Xbyak::Allocator*) ()
#9 0x000003482b605953 in PopCountTest::PopCountTest(int) ()
#10 0x000003482b6049e0 in putCPUinfo() ()
#11 0x000003482b604bf6 in main ()
Are these CPUs fundamentally incompatible with your code or is there a AMD-specific bug?
If you change bls/bls.go as the followings, then does it run?
--- a/bls/bls.go
+++ b/bls/bls.go
@@ -14,7 +14,7 @@ package bls
#cgo darwin,amd64 LDFLAGS:-L${SRCDIR}/lib/darwin/amd64
#cgo darwin,arm64 LDFLAGS:-L${SRCDIR}/lib/darwin/arm64
#cgo windows,amd64 LDFLAGS:-L${SRCDIR}/lib/windows/amd64
-#cgo openbsd,amd64 LDFLAGS:-L${SRCDIR}/lib/openbsd/amd64
+#cgo openbsd,amd64 LDFLAGS:-L${SRCDIR}/lib/linux/amd64
I installed OpenBSD 6.8 and test it. It seems good.
Though xbyak/sample/test_util.cpp
shows an error, but static_code
mode does not use mprotect, so it is no problem.
mcl ; master 4407af6 update doc
bls ; master 3005a32 MCLBN_FORCE_EXPORT exports mcl-functions
bls-eth-go-binary ; master d782bdf remove android binaries for arm64-v8a and x86_64
% uname -a
OpenBSD openbsd.localdomain 6.8 GENERIC.MP#98 amd64
% clang --version
OpenBSD clang version 10.0.1
Target: amd64-unknown-openbsd6.8
% go version
go version go1.15.2 openbsd/amd64
% cat /var/run/dmesg.boot | grep cpu0
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz, 3408.35 MHz, 06-5e-03
cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,SS,SSE3,PCLMUL,SSSE3,FMA3,CX16,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,HV,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,RDSEED,ADX,SMAP,CLFLUSHOPT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES,MELTDOWN
-----------------------------------------------------------------------------
xbyak/sample% clang++ test_util.cpp -I ../ && ./a.out
64bit
vendor intel
mmx mmx2 cmov sse sse2 sse3 ssse3 sse41 sse42 popcnt aesni rdtscp osxsave(xgetvb) pclmulqdq avx fma avx2 bmi1 bmi2 lzcnt prefetchw enh_rep rdrand adx rdseed smap f16c movbe
terminating with uncaught exception of type Xbyak::Error: can't protect
Abort
-----------------------------------------------------------------------------
bls-eth-go-binary% gmake CXX=clang++
clang++ -c -o obj/bls_c384_256.o ../bls/src/bls_c384_256.cpp -std=c++03 -O3 -fno-exceptions -fno-rtti -fno-threadsafe-statics -fno-stack-protector -DNDEBUG -DMCL_DONT_USE_OPENSSL -DMCL_LLVM_BMI2=0 -DMCL_USE_LLVM=1 -DMCL_USE_VINT -DMCL_SIZEOF_UNIT=8 -DMCL_VINT_FIXED_BUFFER -DMCL_MAX_BIT_SIZE=384 -DCYBOZU_DONT_USE_EXCEPTION -DCYBOZU_DONT_USE_STRING -D_FORTIFY_SOURCE=0 -I../bls/include -I../mcl/include -DBLS_ETH -DBLS_SWAP_G -DMCL_STATIC_CODE -DMCL_DONT_USE_XBYAK -fPIC
clang++ -c -o obj/fp.o ../mcl/src/fp.cpp -std=c++03 -O3 -fno-exceptions -fno-rtti -fno-threadsafe-statics -fno-stack-protector -DNDEBUG -DMCL_DONT_USE_OPENSSL -DMCL_LLVM_BMI2=0 -DMCL_USE_LLVM=1 -DMCL_USE_VINT -DMCL_SIZEOF_UNIT=8 -DMCL_VINT_FIXED_BUFFER -DMCL_MAX_BIT_SIZE=384 -DCYBOZU_DONT_USE_EXCEPTION -DCYBOZU_DONT_USE_STRING -D_FORTIFY_SOURCE=0 -I../bls/include -I../mcl/include -DBLS_ETH -DBLS_SWAP_G -DMCL_STATIC_CODE -DMCL_DONT_USE_XBYAK -fPIC
clang++ -c -o obj/base64.o ../mcl/src/base64.ll -std=c++03 -O3 -fno-exceptions -fno-rtti -fno-threadsafe-statics -fno-stack-protector -DNDEBUG -DMCL_DONT_USE_OPENSSL -DMCL_LLVM_BMI2=0 -DMCL_USE_LLVM=1 -DMCL_USE_VINT -DMCL_SIZEOF_UNIT=8 -DMCL_VINT_FIXED_BUFFER -DMCL_MAX_BIT_SIZE=384 -DCYBOZU_DONT_USE_EXCEPTION -DCYBOZU_DONT_USE_STRING -D_FORTIFY_SOURCE=0 -I../bls/include -I../mcl/include -DBLS_ETH -DBLS_SWAP_G -DMCL_STATIC_CODE -DMCL_DONT_USE_XBYAK -fPIC
clang++: warning: argument unused during compilation: '-I ../bls/include' [-Wunused-command-line-argument]
clang++: warning: argument unused during compilation: '-I ../mcl/include' [-Wunused-command-line-argument]
warning: overriding the module target triple with amd64-unknown-openbsd6.8 [-Woverride-module]
1 warning generated.
gmake -C ../mcl obj/static_code.o
gmake[1]: Entering directory 'mcl'
clang++ -o src/dump_code src/dump_code.cpp src/fp.cpp -g -I include -DMCL_DUMP_JIT -DMCL_MAX_BIT_SIZE=384 -DMCL_DONT_USE_OPENSSL -DMCL_USE_VINT -DMCL_SIZEOF_UNIT=8 -DMCL_VINT_FIXED_BUFFER
src/dump_code > src/static_code.asm
nasm -felf64 -o obj/static_code.o src/static_code.asm
gmake[1]: Leaving directory 'mcl'
mkdir -p bls/lib/openbsd/amd64
ar r bls/lib/openbsd/amd64/libbls384_256.a obj/bls_c384_256.o obj/fp.o obj/base64.o ../mcl/obj/static_code.o
bls-eth-go-binary% go test ./bls
ok github.com/herumi/bls-eth-go-binary/bls 1.592s
---
As you say, the problem seems AMD-based computers. I consider the reason.
Ah I understand your meaning in https://github.com/herumi/bls-eth-go-binary/issues/31#issuecomment-747151355 now and can confirm this works:
alexander@proofofobsd> git --no-pager diff
diff --git a/bls/bls.go b/bls/bls.go
index 273724e..b08ad55 100644
--- a/bls/bls.go
+++ b/bls/bls.go
@@ -14,7 +14,7 @@ package bls
#cgo darwin,amd64 LDFLAGS:-L${SRCDIR}/lib/darwin/amd64
#cgo darwin,arm64 LDFLAGS:-L${SRCDIR}/lib/darwin/arm64
#cgo windows,amd64 LDFLAGS:-L${SRCDIR}/lib/windows/amd64
-#cgo openbsd,amd64 LDFLAGS:-L${SRCDIR}/lib/openbsd/amd64
+#cgo openbsd,amd64 LDFLAGS:-L${SRCDIR}/lib/linux/amd64
#cgo freebsd,amd64 LDFLAGS:-L${SRCDIR}/lib/linux/amd64
typedef unsigned int (*ReadRandFunc)(void *, void *, unsigned int);
int wrapReadRandCgo(void *self, void *buf, unsigned int n);
alexander@proofofobsd> go test ./bls
ok github.com/herumi/bls-eth-go-binary/bls 3.730s
alexander@proofofobsd> uname -prs
OpenBSD 6.8 amd64
I have changed my packages to use this pre-build library for now but would obviously prefer to build from source. I hope you can find a solution and am of course willing to test or help if required. Thank you so far.
I want to know what causes Illegal instruction of ./src/dump_code.
Could you run the following command?
cd mcl
gdb ./src/dump_code
r
bt
I want to see the last message around the Illegal instruction (core dumped)
.
vendor amd
mmx mmx2 cmov sse sse2 sse3 ssse3 sse41 sse42 popcnt aesni rdtscp osxsave(xgetvb) pclmulqdq avx lzcnt prefetchw
vendor amd
mmx mmx2 cmov sse sse2 sse3 ssse3 sse41 sse42 popcnt aesni rdtscp osxsave(xgetvb) pclmulqdq avx fma bmi1 lzcnt prefetchw f16c
Oh, your CPU does not have bmi2. It may be the reason.
I want to know what causes Illegal instruction of ./src/dump_code.
Could you run the following command?
cd mcl gdb ./src/dump_code r bt
I want to see the last message around the
Illegal instruction (core dumped)
.
Here is output using mcl latest master:
Program received signal SIGILL, Illegal instruction.
0x00000f10bd8c93f4 in ?? ()
(gdb) bt
#0 0x00000f10bd8c93f4 in ?? ()
#1 0x00000f10bd8c93d2 in ?? ()
#2 0x0000000000000001 in ?? ()
#3 0x00007f7ffffeb2c8 in ?? ()
#4 0x00000f11807c09f0 in _dl_run_all_dtors () at /usr/src/libexec/ld.so/loader.c:186
#5 0x00007f7ffffeb2d8 in ?? ()
#6 0x00007f7ffffea860 in ?? ()
#7 0x00000f11808bd020 in ?? () from /usr/libexec/ld.so
#8 0x00000f0eaddab3a5 in mcl::fp::Op::toMont (this=0xcccccccccccc0d74, y=0xfa6501d3b4c24, x=0x1c334c5d5b415f41)
at include/mcl/op.hpp:370
Backtrace stopped: previous frame inner to this frame (corrupt stack?)
Thank you for the useful information. Could you try this version of https://github.com/herumi/mcl/commit/8ecd8cf99fc28468a7001c6fefee7a8977284c7e ?
That version fixes the problem, full output here.
New binary also builds and tests OK:
goos: openbsd
goarch: amd64
pkg: github.com/herumi/bls-eth-go-binary/bls
BenchmarkPairing-8 564 2222209 ns/op
BenchmarkSignHash-8 705 1694639 ns/op
BenchmarkVerifyHash-8 307 3989177 ns/op
BenchmarkVerifyAggreageteHash-8 13 90283057 ns/op
BenchmarkNaieveMultiVerify-8 1 1532228068 ns/op
BenchmarkMultiVerify-8 8 143120830 ns/op
PASS
ok github.com/herumi/bls-eth-go-binary/bls 16.491s
Thank you!
Thank you for the verification.
I have been unable to build static bls libraries for OpenBSD since the MCL_STATIC_CODE commits were added to the repository. The build log does not contain errors and succesfully creates a binary, but unfortunately when I run the 'go test ./bls -bench' command from the instructions, a large number of undefined symbol errors are outputted.
Going over the compilation steps reveals a clue. Unlike the binaries produced by my Linux machine, the src/dump_code produced on my OpenBSD laptop does not produce output and as such nothing to written with the
src/dump_code > src/static_code.asm
command.I have been unable to ascertain why that is so far and hope you have an idea.