Linker ignores library

[Kambis]

I am trying to use Intel MKL in pony but I have problems forcing pony to link to libm which is required by mkl_core.

Here is an example:

use "path:/opt/intel/compilers_and_libraries_2016.0.109/linux/mkl"
use "path:/opt/intel/compilers_and_libraries_2016.0.109/linux/compiler/lib/intel64_lin"
use "lib:mkl_rt"
use "lib:imf"
use "collections"

use @cblas_dcopy[None](n: I64, x: Pointer[F64] tag, incX: I64, y: Pointer[F64] tag, incY: I64)

use @LAPACKE_zgeev[I64](matrix_order: I32, jobvl:U8, jobvr:U8, n: I64, a: Pointer[F64] tag, lds: I64, w : Pointer[F64] tag, vl: Pointer[F64] tag, ldvl: I64, vr: Pointer[F64] tag, ldvr: I64)

primitive LapackRowMajor  fun apply(): I32 => 101
primitive LapackColMajor  fun apply(): I32 => 102

primitive EigenvaluesComplex
  fun apply(m: Array[F64] box) : Array[F64] ? =>

    let n = ((m.size().f64()/2).sqrt()+0.5).floor().usize()

    if (2*n*n)!= m.size() then error end

    var w  : Array[F64] = Array[F64].undefined(2*n)
    var vl : Array[F64] = Array[F64].undefined(2*n)
    var vr : Array[F64] = Array[F64].undefined(2*n)
    var a : Array[F64] = Array[F64].undefined(m.size())
    @cblas_dcopy(m.size().i64(),m.cstring(),1,a.cstring(),1)

    // -- N = 78 , V = 86
    let info : I64 = @LAPACKE_zgeev(LapackRowMajor(),78,78,n.i64(),a.cstring(),n.i64(),w.cstring(),vl.cstring(),n.i64(),vr.cstring(),n.i64())
    if info!=0 then error end

    w

actor Main
  let _env: Env

  fun tag init_mkl() =>
    let ilp64: I32 = 1
    let threading_intel: I32 = 0
    @MKL_Set_Interface_Layer[I32](ilp64)
    @MKL_Set_Threading_Layer[I32](threading_intel)

  new create(env: Env) =>
    _env = env
    init_mkl()

    var m : Array[F64] = [1,0,2,0,3,0,4,0]
    try
      let v = EigenvaluesComplex(m)
      for k in Range(0,4) do
        _env.out.print(v(k).string())
      end
    end

It compiles but I get the runtime error symbol lookup error: /opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/lib/intel64/libmkl_core.so: undefined symbol: log10

Using ldd I can see that the executable is not linked to libm. if I include log10 use @log10[F64](x: F64) and make nontrivial use of it, then everything is fine.

How can I force ponyc to link a library in such a case?

EDIT: As a workaround I changed a linker option in genexe.c from "-lm" to ""-Wl,--whole-archive -l:libm.a -Wl,--no-whole-archive".

EDIT2: Alternatively adding "-Wl,--no-as-needed" before used libraries are linked also does the job. It would be nice to specify such options with the use command.

[SeanTAllen]

@sylvanc this is out of my current wheelhouse. anything you can weigh in with?

[rurban]

How about this?

use "lib:m"
use "path:/opt/intel/compilers_and_libraries_2016.0.109/linux/mkl"
use "path:/opt/intel/compilers_and_libraries_2016.0.109/linux/compiler/lib/intel64_lin"
use "lib:mkl_rt"
use "lib:imf"

[Kambis]

@rurban, I get the same error,

symbol lookup error: /opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/lib/intel64/libmkl_core.so: undefined symbol: log10

The reason probably lies in mkl_rt and how it works. --no-as-needed seems to be a necessary option when linking against it: https://software.intel.com/en-us/articles/intel-mkl-link-line-advisor

[sylvanc]

It looks like Pony needs library linking annotations. I think the right solution is a comma separated list, for example:

use "lib:m@noasneeded"
use "lib:something@wholearchive,noasneeded"

The second example is silly, since there's no need to have both flags, but I include it as an example of what I mean by delimited annotations.

Other than noasneeded and wholearchive, are there any other linker annotations that are needed?

[Kambis]

'static' could also be useful. I would also appreciate the ability to use environment variables in path command.

[tmmcguire]

How about this instead:

use "lib:m@--no-as-needed"
use "lib:something@--whole-archive,--no-as-needed"
use "lib:xyzzy@--static"

That way, the individual strings can be passed to the linker rather than having a look-up table to interpret the annotations?

On the other hand, it's a potential security hole, since the linker arguments are passed to a shell command or could otherwise be exploited. Compiling a source file shouldn't result in all of your files being infected with scabies.

[Kambis]

@tmmcguire That is not a good idea because some flags need to be disabled afterwards, for example --no-as-needed should be followed by --as-needed before linking to other libraries.

[SeanTAllen]

@sylvanc did we ever finalize this?

[jemc]

This can be considered to be final. @sylvanc's comment (https://github.com/ponylang/ponyc/issues/665#issuecomment-203606308) should be what we implement for this.

[rurban]

plus static, right?

[sylvanc]

As in static to prefer a static library even when a dynamic one is present? If so, yes, I think that's a good idea.