Provide binaries compatible with older systems

[nmattia]

The go binaries are built with a very recent Ubuntu image (ubuntu-latest). This can cause issues when artifacts need to be used on older systems.

In my case, I'm trying to use goreleaser with goreleaser-cross because of CGO. The CGO build on Linux fails during linking with the following undefined (c++) reference: std::__throw_bad_array_new_length() (goreleaser-cross uses an old debian image).

After having wrestled with goreleaser-cross I've decided to simply build bls-go-binary with an older libc etc which fixed the issue:

https://github.com/herumi/bls-go-binary/commit/6a07c7097d792ece6512d7688ec3fbba351dd3f5

In general it's a good practice to build artifacts on old systems to avoid compatibility issues. Would it make sense to do the same here? Are there benefits to using the latest ubuntu? Happy to submit a PR otherwise.

[herumi]

Thank you for the good idea. There are two ubuntu-latest in release.yml. Should we change both?

[nmattia]

@herumi I think only the one on line 42 matters, the other one might as well stay on ubuntu latest (unless you find this confusing). I'll submit a PR!

[nmattia]

Quick question: I was facing a similar issue with a windows build (rather, cross compiling from debian to windows), though changing the build from windows-latest to windows-2019 did not fix this issue.

I'm getting a few undefined references when using x86_64-w64-mingw32-clang(++):

...
ld.lld: error: undefined symbol: std::__throw_logic_error(char const*)
>>> referenced by libbls384_256.a(fp.o):(std::__cxx11::basic_string<char, std::char_trait
s<char>, std::allocator<char>>::basic_string(char const*, std::allocator<char> const&) (.
constprop.0))
>>> referenced by libbls384_256.a(fp.o):(mcl::fp::FpGenerator::mkLabel[abi:cxx11](char co
nst*, int) const (.constprop.0))
...
ld.lld: error: undefined symbol: std::tr1::__detail::__prime_list
>>> referenced by libbls384_256.a(fp.o):(.refptr._ZNSt3tr18__detail12__prime_listE)

Do you know where those symbols come from?

$ nm -C x86_64-w64-mingw32/lib/libc++abi.a | grep throw_logic_error
<nothing>

[herumi]

1. Do you get the same error when you make clean && make on your machine?
2. Would be it fixed if you changed the next line from -std=c++03 to -std=c++11?

MIN_CFLAGS+=-std=c++03

https://github.com/herumi/bls/blob/master/Makefile.onelib#L13

[herumi]

It worked correctly in my environment with the original unmodified binary of the original release branch. .

git branch -v
* release 589966a fix(release): added libs for windows

$ gcc --version
gcc.exe (Rev6, Built by MSYS2 project) 12.2.0

$ clang --version
clang version 15.0.5
Target: x86_64-w64-windows-gnu
Thread model: posix

$ uname -a
MINGW64_NT-10.0-22631 3.4.9.x86_64 2023-09-15 12:15 UTC x86_64 Msys

[nmattia]

Do you get the same error when you make clean && make on your machine?

I can't easily try this out on my machine (macOS) and in Docker (ghcr.io/goreleaser/goreleaser-cross:v1.22.3) the build fails fairly early:

/go# CC=/llvm-mingw/bin/x86_64-w64-mingw32-gcc CXX=/llvm-mingw/bin/x86_64-w64-mingw32-g++ make -C ./src/bls -f Makefile.onelib LIB_DIR=./

when setting -std=c++11 on the bls-go-binary GitHub actions, I get a build failure on Linux already:

src//bls/mcl/include/cybozu/hash.hpp:48:10: fatal error: 'functional' file not found
#include <functional>
         ^~~~~~~~~~~~
1 error generated.
make[2]: *** [/home/runner/work/bls-go-binary/bls-go-binary/src/bls/Makefile.onelib:115: obj/bls_c384_256.o] 

I'm actually surprised; judging from the errors I got originally on Windows I assumed c++11 was in use already!

I'm gonna shelve the Windows builds for now and revisit in a couple of days

[herumi]

Sorry for the late reply.

fatal error: 'functional' file not found

The error means that there is no standard including headers. Can you run sudo apt install g++-multilib?

[herumi]

# bls-go-binary
$ git br -v
  dev     6902be2 Merge remote-tracking branch 'origin/dev'
  master  19c7242 [behind 105] update submodule again
* release 589966a fix(release): added libs for windows

$ nm bls/lib/windows/amd64/libbls384_256.a |grep __prime_list|c++filt
0000000000000000 r .rdata$.refptr._ZNSt3tr18__detail12__prime_listE
0000000000000000 R .refptr._ZNSt3tr18__detail12__prime_listE
                 U std::tr1::__detail::__prime_list

$ make clean
$  rm bls/lib/windows/amd64/libbls384_256.a
# change c++03 to c++11
$ (cd src/bls ; git diff)
diff --git a/Makefile.onelib b/Makefile.onelib
index de38e66..24f92c1 100644
--- a/Makefile.onelib
+++ b/Makefile.onelib
@@ -10,7 +10,8 @@ UNIT?=$(shell expr $(BIT) / 8)
 THIS_FILE := $(realpath $(firstword $(MAKEFILE_LIST)))

 MIN_CFLAGS=-O3 -fno-exceptions -fno-threadsafe-statics -fno-rtti -fno-stack-protector -DNDEBUG -DMCL_DONT_USE_OPENSSL -DMCL_SIZEOF_UNIT=$(UNIT) -DMCL_MAX_BIT_SIZE=384 -DCYBOZU_DONT_USE_EXCEPTION -DCYBOZU_DONT_USE_STRING -D_FORTIFY_SOURCE=0 -I$(BLS_DIR)/include -I$(MCL_DIR)/include $(ETH_CFLAGS) $(CFLAGS_USER)
-MIN_CFLAGS+=-std=c++03
+#MIN_CFLAGS+=-std=c++03
+MIN_CFLAGS+=-std=c++11
 OBJ_DIR=obj
 OBJS=$(OBJ_DIR)/bls_c384_256.o $(OBJ_DIR)/fp.o

$ make -j
$  nm bls/lib/windows/amd64/libbls384_256.a |grep __prime_list|c++filt

I changed the option, the symbol vanished, so I'll apply it.

[herumi]

How about https://github.com/herumi/bls-go-binary/tree/dev ?

[herumi]

It has something wrong. Please wait a moment to check.

[herumi]

Could you check https://github.com/herumi/bls-go-binary/commit/0ddf7ee327e65241902043fd84761aad04f12457 ?

[nmattia]

@herumi I'm actually getting build failures on Linux now during the "release" (same functional missing I had on Windows here):

src//bls/mcl/include/cybozu/hash.hpp:48:10: fatal error: 'functional' file not found

include

     ^~~~~~~~~~~~

1 error generated. link

That being said, if I skip all the non-Windows steps, I do get a binary out of it: https://github.com/nmattia/bls-go-binary/actions/runs/9301981474/job/25601238960

Using that binary in my build I get the following error:

ld.lld: error: undefined symbol: std::__throw_bad_array_new_length() referenced by libbls384_256.a(fp.o):(void Xbyak::LabelManager::define_inner...

which I think is a good step forward, since I think it's the same issue as I had originally on Linux (see original message) and may actually be fixed by building the windows lib on an older Windows (with older libc++)!

Unfortunately I can't test this out because the bls-go-binary windows build appears to be failing on windows server 2019:

C:\Users\RUNNER~1\AppData\Local\Temp\ccz7Hb1m.s:30: Error: invalid register for .seh_savexmm link

Hope this helps!

[herumi]

Error: invalid register for .seh_savexmm

The error seems to be caused by AVX-512 handling. It is an experimental feature, so please disable it.

release.yml

#make -C ./src/bls -f Makefile.onelib LIB_DIR=./
make -C ./src/bls -f Makefile.onelib LIB_DIR=./ MCL_MSM=0

[nmattia]

It is an experimental feature, so please disable it.

Unfortunately now I seem to get MCL-related errors:

ld.lld: error: undefined symbol: mcl::msm::initMsm(mcl::CurveParam const&, mcl::msm::Param
 const*)
>>> referenced by libbls384_256.a(bls_c384_256.o):(mclBn_init)
>>> referenced by libbls384_256.a(bls_c384_256.o):(blsInit)

alongside some C++11 errors still...

ld.lld: error: undefined symbol: std::__cxx11::basic_string<char, std::char_traits<char>, 
std::allocator<char>>::_M_create(unsigned long long&, unsigned long long)
>>> referenced by libbls384_256.a(fp.o):(std::__cxx11::basic_string<char, std::char_traits
<char>, std::allocator<char>>::basic_string(char const*, std::allocator<char> const&) (.is
ra.397))

[nmattia]

@herumi should we merge https://github.com/herumi/bls-go-binary/pull/32 already? Windows is not a top priority for me, though I'd be happy to revisit later on

[herumi]

If we use -std=c++11 option to avoid the error std::__throw_bad_array_new_length, then Clang can't find functional header. I don't have an old environment. It takes time.

[herumi]

https://github.com/herumi/bls-go-binary/actions/runs/9316922774/job/25646157813#step:8:434

clang++ -c src//bls/src/bls_c384_256.cpp -o obj/bls_c384_256.o -O3 -fno-exceptions -fno-threadsafe-statics -fno-rtti -fno-stack-protector -DNDEBUG -DMCL_DONT_USE_OPENSSL -DMCL_SIZEOF_UNIT=8 -DMCL_MAX_BIT_SIZE=384 -DCYBOZU_DONT_USE_EXCEPTION -DCYBOZU_DONT_USE_STRING -D_FORTIFY_SOURCE=0 -Isrc//bls/include -Isrc//bls/mcl/include   -std=c++11 -fpic -target aarch64 -DMCL_USE_LLVM=1
In file included from src//bls/src/bls_c384_256.cpp:3:
In file included from src//bls/src/bls_c_impl.hpp:8:
In file included from src//bls/mcl/include/mcl/impl/bn_c_impl.hpp:13:
In file included from src//bls/mcl/include/mcl/bls12_381.hpp:11:
In file included from src//bls/mcl/include/mcl/bn.hpp:11:
In file included from src//bls/mcl/include/mcl/fp_tower.hpp:9:
In file included from src//bls/mcl/include/mcl/fp.hpp:28:
src//bls/mcl/include/cybozu/hash.hpp:48:10: fatal error: 'functional' file not found
#include <functional>
         ^~~~~~~~~~~~

[herumi]

https://github.com/herumi/bls-go-binary/releases/tag/v1.34.0 is built on Ubuntu 20.04 with c++03.

[herumi]

The functional header loss problem occurs when making binaries for Aarch64 on Linux with -std=c++11. So, it may work if you build only in a Windows environment with c++11 and MCL_MSM=0.

[nmattia]

So, it may work if you build only in a Windows environment with c++11 and MCL_MSM=0.

Nice! I'll give it a try. That'd still cause the MCL error on windows though right?

It takes time.

This is much, much appreciated!

[herumi]

That'd still cause the MCL error on windows though right?

There is no error in my environment (both v1.33.0 and v1.34.0), so it is hard to find out what is causing this in your environment.

git branch -v
* release c1d9b11 fix(release): added libs for windows

/c/prog/bls-go-binary
shigeo@fmv4 MINGW64 /c/prog/bls-go-binary
$ go test -v -count=1 ./bls
=== RUN   TestSign
sec:2e6c7cb0a8f5d34bd7fa1f96056f818c237b3ecdde5dc49aeeb8634df0ac4b02
pub:50091c428511f2b3ff4c7a01b7f153254e5ed2d72b2ebb8bd0ce4a7e0a26f4340077a6510e963129aa71cc79bb44f1160b8a07dffee8159695223d018bf70b0460bd1ed75619d3e90e309029ebf910a90eb580339e92f5589c9226afad328a89
0. sign(abc)=b01d9eb4ec66a3385e8665ca86566235df138fc0c15663b31581e7aedfe6d387ec97e39ea643245b86bb66ee11e88695
1. sign(def)=a9e7dbbe34a0dfb9fe33afcf1546311c58bb4327c5c0ca265aa5958ed1e5123add0fbee7360bf88821077bb46cb2cc0b
2. sign(123)=7148dcc9845bcd7751f3a7e577be77fa7d06cb8c1c6b18878cecfdc523d3216b3b42e97c936362e68a797d70d1825381
--- PASS: TestSign (0.00s)
=== RUN   TestPairing
--- PASS: TestPairing (0.01s)
=== RUN   TestCast
--- PASS: TestCast (0.00s)
=== RUN   TestDFINITY
--- PASS: TestDFINITY (0.00s)
PASS
ok      github.com/herumi/bls-go-binary/bls     0.397s

[herumi]

I can't reproduce your Windows environment. Please tell me how to cause an error in detail.