status-im / nim-bearssl

BearSSL wrapper in Nim
Other
40 stars 8 forks source link

WIP: Let bearssl be trimmed during linking to static lib #54

Open iffy opened 9 months ago

iffy commented 9 months ago

This addresses #53

I found the key in bearssl/csources/README.txt:

Using the library means writing some application code that invokes it, and linking with the static library. The header files are all in the inc directory; copy them wherever makes sense (e.g. in the /usr/local/include directory). The library itself (libbearssl.a) is what you link against.

So with the code in this PR, I do the following on macOS:

(cd bearssl/csources && make)
nim c --path:"$(pwd)" -f tests/samp.nim
tests/samp "prove it runs"
nm tests/samp | grep br_

And get this output:

$ nim c --path:"$(pwd)" -f tests/samp.nim
Hint: used config file '/Users/matt/.choosenim/toolchains/nim-1.6.18/config/nim.cfg' [Conf]
Hint: used config file '/Users/matt/.choosenim/toolchains/nim-1.6.18/config/config.nims' [Conf]
Hint: used config file '/Users/matt/lib/nim-bearssl/nim.cfg' [Conf]
Hint: used config file '/Users/matt/lib/nim-bearssl/config.nims' [Conf]
Hint: used config file '/Users/matt/lib/nim-bearssl/tests/nim.cfg' [Conf]
...........................................................................................
CC: ../../../.choosenim/toolchains/nim-1.6.18/lib/std/private/digitsutils.nim
CC: ../../../.choosenim/toolchains/nim-1.6.18/lib/system/dollars.nim
CC: ../../../.choosenim/toolchains/nim-1.6.18/lib/system/io.nim
CC: ../../../.choosenim/toolchains/nim-1.6.18/lib/system.nim
CC: ../../../.choosenim/toolchains/nim-1.6.18/lib/posix/posix.nim
CC: ../../../.choosenim/toolchains/nim-1.6.18/lib/pure/times.nim
CC: ../../../.choosenim/toolchains/nim-1.6.18/lib/pure/os.nim
CC: samp.nim
Hint:  [Link]
Hint: gc: refc; opt: none (DEBUG BUILD, `-d:release` generates faster code)
59234 lines; 1.069s; 75.652MiB peakmem; proj: /Users/matt/lib/nim-bearssl/tests/samp.nim; out: /Users/matt/lib/nim-bearssl/tests/samp [SuccessX]

$ tests/samp "prove it runs"
[66, 113, 209, 141, 21, 155, 157, 198, 213, 30, 180, 244, 25, 63, 196, 233]

$ nm tests/samp | grep br_
00000001000132d0 S _br_md5_IV
0000000100012769 T _br_md5_init
0000000100012829 T _br_md5_out
0000000100012452 T _br_md5_round
000000010001293a T _br_md5_set_state
0000000100012912 T _br_md5_state
0000000100012797 T _br_md5_update
00000001000140b0 S _br_md5_vtable
000000010001241c T _br_range_dec32le
0000000100012437 T _br_range_enc32le

It works! (At least for MD5).

But I don't know the correct way to change this library to either build the static lib or get users to build the static lib themselves. Thoughts?

arnetheduck commented 9 months ago

Right, this reminds me of the following: https://stackoverflow.com/questions/53414293/gcc-differently-treats-an-object-and-a-static-library-regarding-undefined-symbol - ie there's a difference of linking objects vs static libraries - maybe there's a way to get the static-library-like linking even when using object files? something to investigate..

iffy commented 9 months ago

How about this latest change? This lets library users keep the existing behavior:

nim c myprog.nim

Or compile with their own libbearssl.a

nim c -d:bearStaticLibPath=/path/to/libbearssl.a myprog.nim
arnetheduck commented 2 months ago

How about this latest change? This lets library users keep the existing behavior:

the thing missing here above all is the step that ensures that the static library corresponds to the actual source code of bearssl and is compiled using the same options, compiler and flags as the nim code (optimization flags, LTO etc) - this is the difficulty when working with .a files without the powers that make has, namely to keep track of dependencies - nim does a poo job of this even with .compile and with the static linking option, we go even further away from "just works" because of the manual steps involved.

the way one would typically get the same end result with a modern toolchain is to do LTO + whole-program optimisation or possibly function/data sections (-ffunction-sections and friends) + link-time garbage collection, which neatly works around nim's deficiences but puts some additional constraints on the end user

iffy commented 2 months ago

Thanks for taking a look! I'll admit that I don't have recent expertise with compiling, so if you think it's not a good idea, that's fine. I'm not clear if you're worried about things being broken because of this change, or if you're just worried they won't be optimal.

we go even further away from "just works" because of the manual steps involved.

This PR makes linking to a .a opt-in. So this library still "just works" as-is. If someone wants to link to a .a, they'll be responsible for figuring out how to do that and must explicitly decide to do so (with -d:bearStaticLibPath=/path/to/libbearssl.a). This PR makes it possible, but doesn't cause it to happen by default.

the way one would typically get the same end result with a modern toolchain...

By "same end result" here, do you mean the trimming unused bearssl symbols magic that BearSSL does?

Again, thanks for taking the time (and for publishing this library).