Linking issue on OpenBSD

[thoferon]

Hi,

I can't link code using cryptonite on OpenBSD anymore. After using git bisect, the commit introducing the error seems to be 3c89f0d0b7141eeac2c425724b1fdf41789a6cf2.

Unfortunately, I don't know how to fix the issue so I can't send a PR but maybe somebody can help me?

Here is the error: (things defined in cbits/decaf/ed448goldilocks/decaf_tables.c)

    /home/thoferon/tmp/cryptonite-test/.stack-work/install/x86_64-openbsd/nightly-2017-09-06/8.2.1/lib/x86_64-openbsd-ghc-8.2.1/cryptonite-0.23-5YLjX9JzI8E1xVQ8Df1upo/libHScryptonite-0.23-5YLjX9JzI8E1xVQ8Df1upo.a(decaf.o): In function `cryptonite_decaf_448_direct_scalarmul':
    (.text+0x51d7): undefined reference to `cryptonite_decaf_448_point_base'
    /home/thoferon/tmp/cryptonite-test/.stack-work/install/x86_64-openbsd/nightly-2017-09-06/8.2.1/lib/x86_64-openbsd-ghc-8.2.1/cryptonite-0.23-5YLjX9JzI8E1xVQ8Df1upo/libHScryptonite-0.23-5YLjX9JzI8E1xVQ8Df1upo.a(decaf.o):(.data.rel+0x0): undefined reference to `cryptonite_decaf_448_precomputed_base_as_fe'
    /home/thoferon/tmp/cryptonite-test/.stack-work/install/x86_64-openbsd/nightly-2017-09-06/8.2.1/lib/x86_64-openbsd-ghc-8.2.1/cryptonite-0.23-5YLjX9JzI8E1xVQ8Df1upo/libHScryptonite-0.23-5YLjX9JzI8E1xVQ8Df1upo.a(decaf.o):(.data.rel.ro+0x0): undefined reference to `cryptonite_decaf_448_precomputed_wnaf_as_fe'
    collect2: ld returned 1 exit status

I don't know if it helps but this doesn't seem normal to me:

$ ar -x .stack-work/install/x86_64-openbsd/nightly-2017-09-06/8.2.1/lib/x86_64-openbsd-ghc-8.2.1/cryptonite-0.23-5YLjX9JzI8E1xVQ8Df1upo/libHScryptonite-0.23-5YLjX9JzI8E1xVQ8Df1upo.a decaf_tables.o
$ objdump -x decaf_tables.o

decaf_tables.o:     file format elf64-x86-64
decaf_tables.o
architecture: i386:x86-64, flags 0x00000000:

start address 0x0000000000000000

Sections:
Idx Name          Size      VMA               LMA               File off  Algn
  0 .text         00000000  0000000000000000  0000000000000000  00000040  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
  1 .data         00000000  0000000000000000  0000000000000000  00000040  2**2
                  CONTENTS, ALLOC, LOAD, DATA
  2 .bss          00000000  0000000000000000  0000000000000000  00000040  2**2
                  ALLOC
  3 .rodata       00005500  0000000000000000  0000000000000000  00000040  2**5
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
SYMBOL TABLE:
no symbols

Thanks for any help.

[ocheron]

OpenBSD was one of the platforms I tested, and I'll see if I can reproduce this. We'll also need to compare OS version and C compiler.

[ocheron]

My setup is clearly different because I used binary distributions I found at that time: OpenBSD 6.0, GHC 7.10.3 and default C compiler based on GCC 4.2.1.

With commit 3c89f0d the result I get is better looking:

$ ar -x dist/build/libHScryptonite-0.23-850bT9zWDuW7BKbhOPmren.a decaf_tables.o
$ objdump -x decaf_tables.o

decaf_tables.o:     file format elf64-x86-64
decaf_tables.o
architecture: i386:x86-64, flags 0x00000010:
HAS_SYMS
start address 0x0000000000000000

Sections:
Idx Name          Size      VMA               LMA               File off  Algn
  0 .text         00000000  0000000000000000  0000000000000000  00000040  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
  1 .data         00000000  0000000000000000  0000000000000000  00000040  2**2
                  CONTENTS, ALLOC, LOAD, DATA
  2 .bss          00000000  0000000000000000  0000000000000000  00000040  2**2
                  ALLOC
  3 .rodata       00005500  0000000000000000  0000000000000000  00000040  2**5
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
SYMBOL TABLE:
0000000000000000 l    df *ABS*  0000000000000000 decaf_tables.c
0000000000000000 l    d  .text  0000000000000000 .text
0000000000000000 l    d  .data  0000000000000000 .data
0000000000000000 l    d  .bss   0000000000000000 .bss
0000000000000000 l    d  .rodata        0000000000000000 .rodata
0000000000000000 g     O .rodata        0000000000000100 cryptonite_decaf_448_point_base
0000000000000100 g     O .rodata        0000000000003c00 .hidden cryptonite_decaf_448_precomputed_base_as_fe
0000000000003d00 g     O .rodata        0000000000001800 .hidden cryptonite_decaf_448_precomputed_wnaf_as_fe

You could try to touch cbits/decaf/ed448goldilocks/decaf_tables.c and recompile verbosely to see if some warnings explain why you get no symbol at all.

Also please note that later commits like 8be985640270695522700375cfb468c1b226dc3e specifically deals with warnings on OpenBSD. It would be more comfortable to know if you have the exact same symptom with released version.

[thoferon]

Thanks for your response.

I made some progress. After I tried with GHC 7.10.3 as you suggested, I still had the same problem but I noticed that the symbols were in the file you tested dist/build/libHScruptonite-....a(decaf_tables.o) but not in the file I was testing in .cabal-sandbox or .stack-work.

After looking at the logs, it seems that before installing the archive file, Cabal calls strip --strip-unneeded on it which removes the symbols from decaf_tables.o.

Installing
dist/dist-sandbox-c9e3e448/build/libHScryptonite-0.24-ACXNmCWtDPkESnpPN61sqt.a
to
/home/thoferon/tmp/cryptonite-test/cryptonite/.cabal-sandbox/lib/x86_64-openbsd-ghc-7.10.3/cryptonite-0.24-ACXNmCWtDPkESnpPN61sqt/libHScryptonite-0.24-ACXNmCWtDPkESnpPN61sqt.a
Environment: ...
("/usr/bin/strip",["/home/thoferon/tmp/cryptonite-test/cryptonite/.cabal-sandbox/lib/x86_64-openbsd-ghc-7.10.3/cryptonite-0.24-ACXNmCWtDPkESnpPN61sqt/libHScryptonite-0.24-ACXNmCWtDPkESnpPN61sqt.a","--strip-unneeded"])

And indeed,

$ cp dist/dist-sandbox-c9e3e448/build/libHScryptonite-0.24-ACXNmCWtDPkESnpPN61sqt.a .
$ ar -x libHScryptonite-0.24-ACXNmCWtDPkESnpPN61sqt.a decaf_tables.o
$ objdump -x decaf_tables.o

decaf_tables.o:     file format elf64-x86-64
decaf_tables.o
architecture: i386:x86-64, flags 0x00000010:
HAS_SYMS
start address 0x0000000000000000

Sections:
Idx Name          Size      VMA               LMA               File off  Algn
  0 .text         00000000  0000000000000000  0000000000000000  00000040  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
  1 .data         00000000  0000000000000000  0000000000000000  00000040  2**2
                  CONTENTS, ALLOC, LOAD, DATA
  2 .bss          00000000  0000000000000000  0000000000000000  00000040  2**2
                  ALLOC
  3 .rodata       00005500  0000000000000000  0000000000000000  00000040  2**5
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
SYMBOL TABLE:
0000000000000000 l    df *ABS*  0000000000000000 decaf_tables.c
0000000000000000 l    d  .text  0000000000000000 .text
0000000000000000 l    d  .data  0000000000000000 .data
0000000000000000 l    d  .bss   0000000000000000 .bss
0000000000000000 l    d  .rodata        0000000000000000 .rodata
0000000000000000 g     O .rodata        0000000000000100 cryptonite_decaf_448_point_base
0000000000000100 g     O .rodata        0000000000003c00 cryptonite_decaf_448_precomputed_base_as_fe
0000000000003d00 g     O .rodata        0000000000001800 cryptonite_decaf_448_precomputed_wnaf_as_fe

$ strip libHScryptonite-0.24-ACXNmCWtDPkESnpPN61sqt.a --strip-unneeded
$ ar -x libHScryptonite-0.24-ACXNmCWtDPkESnpPN61sqt.a decaf_tables.o
$ objdump -x decaf_tables.o

decaf_tables.o:     file format elf64-x86-64
decaf_tables.o
architecture: i386:x86-64, flags 0x00000000:

start address 0x0000000000000000

Sections:
Idx Name          Size      VMA               LMA               File off  Algn
  0 .text         00000000  0000000000000000  0000000000000000  00000040  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
  1 .data         00000000  0000000000000000  0000000000000000  00000040  2**2
                  CONTENTS, ALLOC, LOAD, DATA
  2 .bss          00000000  0000000000000000  0000000000000000  00000040  2**2
                  ALLOC
  3 .rodata       00005500  0000000000000000  0000000000000000  00000040  2**5
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
SYMBOL TABLE:
no symbols

I honestly don't know what to do now.

[vincenthz]

Maybe a bug in cabal or ghc ? might be also a generic toolchain option that doesn't play nice with whatever code organisation decaf is doing ?

[ocheron]

Yes most likely a bug in binutils.

With a version of Cabal that does --strip-unneeded I managed to get the same error when linking to installed cryptonite-0.24.

Simply moving code like this and the problem goes away:

$ cat < cbits/decaf/ed448goldilocks/decaf_tables.c >> cbits/decaf/ed448goldilocks/decaf.c
$ echo > cbits/decaf/ed448goldilocks/decaf_tables.c

@thoferon can you confirm too?

[thoferon]

Yes, it works when moving the definitions into decaf.c. :+1:

[ocheron]

Thanks, I see that Cabal already has code to avoid stripping in some conditions (haskell/cabal#2339) but it will more effective to fix this in cryptonite as we have a simple solution. I'll send a PR.