Link error when executing 'cargo install dinghy'

[learnopengles]

I'm not actually able to compile and install dinghy on my local machine as the linker command fails:

error: linking with `cc` failed: exit code: 1
  |
  = note: "cc" "-m64" "-L" [truncated, whole bunch of rlibs] "-framework" "MobileDevice" "-l" "iconv" "-framework" "Security" "-framework" "CoreFoundation" "-l" "ssl" "-l" "crypto" "-framework" "CoreFoundation" "-l" "curl" "-l" "z" "-l" "System" "-l" "pthread" "-l" "c" "-l" "m"
  = note: Undefined symbols for architecture x86_64:
  "_iconv", referenced from:
      _git_path_iconv in liblibgit2_sys-ce5bbe4559aebb33.rlib(path.c.o)
     (maybe you meant: _git_path_iconv_init_precompose, _git_path_iconv , _git_path_iconv_clear )
  "_iconv_open", referenced from:
      _git_path_direach in liblibgit2_sys-ce5bbe4559aebb33.rlib(path.c.o)
      _git_path_iconv_init_precompose in liblibgit2_sys-ce5bbe4559aebb33.rlib(path.c.o)
      _git_path_diriter_init in liblibgit2_sys-ce5bbe4559aebb33.rlib(path.c.o)
  "_iconv_close", referenced from:
      _git_path_direach in liblibgit2_sys-ce5bbe4559aebb33.rlib(path.c.o)
      _git_path_iconv_clear in liblibgit2_sys-ce5bbe4559aebb33.rlib(path.c.o)
      _git_path_diriter_free in liblibgit2_sys-ce5bbe4559aebb33.rlib(path.c.o)
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

It seems strange since a standalone cargo project referring just to git2 seems to compile OK. I'm not quite sure what it could be or how to fix it at this point.

Possible refs:

https://github.com/rust-lang/cargo/issues/1032 https://github.com/ogham/exa/issues/67

[kali]

Hi, I could not reproduce here, so it's probably not something that was broken by a third party update.

Which version of Xcode and osx are you running ? do you have anything "git related" installed from brew or ports ? I'm specifically concerned about a libgit2 coming from brew.

FYI, I'm running the latest version os osx and xcode, and libgit2 is not installed by neither brew nor ports.

[learnopengles]

Hi kali,

Here is my config:

macOS 10.12.2 Xcode 8.2.1

I have only ports installed, and I did install git from there to see if that would fix the issue. I uninstalled this and still running into it. Here's my git:

git version 2.10.1 (Apple Git-78)

Here is my active rust toolchain:

stable-x86_64-apple-darwin (default) rustc 1.14.0 (e8a012324 2016-12-16)

Strange! I'll keep digging to see what it could be.

[kali]

all right, first of all, be careful to perform a cargo clean between each try, to make sure we don't have some weird half build state in target/ .

I thing your experiment removing PrivateFrameworks crashes earlier: you explosing because a framework is missing, while the initial git issue was later, during symbol resolution, after actual frameworks and libraries have been found.

The linker will eliminate stuff that is not in use: so on an empty project, by not using anything, the linker discard everything and our missing symbol is not longer an issue.

maybe we could compare a few lines from a cargo build -v. The are the lines compiling libgit2-sys, git2, and cargo-dinghy.

   Compiling libgit2-sys v0.4.6
     Running `rustc /Users/kali/.cargo/registry/src/github.com-1ecc6299db9ec823/libgit2-sys-0.4.6/build.rs --crate-name build_script_build --crate-type bin -g --cfg feature=\"libssh2-sys\" --cfg feature=\"ssh\" --cfg feature=\"https\" -C metadata=e36168b6cc5d8ac9 --out-dir /Users/kali/dev/snips/dinghy/target/debug/build/libgit2-sys-e36168b6cc5d8ac9 --emit=dep-info,link -L dependency=/Users/kali/dev/snips/dinghy/target/debug/deps --extern gcc=/Users/kali/dev/snips/dinghy/target/debug/deps/libgcc-b447f4ed382a4030.rlib --extern cmake=/Users/kali/dev/snips/dinghy/target/debug/deps/libcmake-89406e538eb7d0dd.rlib --extern pkg_config=/Users/kali/dev/snips/dinghy/target/debug/deps/libpkg_config-7cc12d9787dada57.rlib --cap-lints allow`
   Compiling git2 v0.4.4
     Running `rustc /Users/kali/.cargo/registry/src/github.com-1ecc6299db9ec823/git2-0.4.4/src/lib.rs --crate-name git2 --crate-type lib -g --cfg feature=\"libgit2-sys\" --cfg feature=\"default\" --cfg feature=\"ssh\" --cfg feature=\"https\" -C metadata=dbe5af82826f4a0e -C extra-filename=-dbe5af82826f4a0e --out-dir /Users/kali/dev/snips/dinghy/target/debug/deps --emit=dep-info,link -L dependency=/Users/kali/dev/snips/dinghy/target/debug/deps --extern libc=/Users/kali/dev/snips/dinghy/target/debug/deps/liblibc-6ec63c5a0e74a074.rlib --extern bitflags=/Users/kali/dev/snips/dinghy/target/debug/deps/libbitflags-fdf4b7654a6da3e5.rlib --extern libgit2_sys=/Users/kali/dev/snips/dinghy/target/debug/deps/liblibgit2_sys-ce5bbe4559aebb33.rlib --extern url=/Users/kali/dev/snips/dinghy/target/debug/deps/liburl-30c51e30991892cc.rlib --cap-lints allow -L native=/Users/kali/dev/snips/dinghy/target/debug/build/libgit2-sys-e36168b6cc5d8ac9/out/lib -L native=/usr/local/Cellar/libssh2/1.8.0/lib -L native=/usr/local/opt/openssl/lib`
     Running `rustc src/bin/cargo-dinghy.rs --crate-name cargo_dinghy --crate-type bin -g -C metadata=65ef35777aed86a0 --out-dir /Users/kali/dev/snips/dinghy/target/debug --emit=dep-info,link -L dependency=/Users/kali/dev/snips/dinghy/target/debug/deps --extern core_foundation=/Users/kali/dev/snips/dinghy/target/debug/deps/libcore_foundation-19c9346851603dc8.rlib --extern plist=/Users/kali/dev/snips/dinghy/target/debug/deps/libplist-a333c8ce64ae0c14.rlib --extern tempdir=/Users/kali/dev/snips/dinghy/target/debug/deps/libtempdir-c5a9975021308003.rlib --extern libc=/Users/kali/dev/snips/dinghy/target/debug/deps/liblibc-6ec63c5a0e74a074.rlib --extern tempfile=/Users/kali/dev/snips/dinghy/target/debug/deps/libtempfile-7cea1a50ab366cea.rlib --extern cargo=/Users/kali/dev/snips/dinghy/target/debug/deps/libcargo-71f3f2ecff92e18a.rlib --extern core_foundation_sys=/Users/kali/dev/snips/dinghy/target/debug/deps/libcore_foundation_sys-93b96f39493ba0ef.rlib --extern env_logger=/Users/kali/dev/snips/dinghy/target/debug/deps/libenv_logger-c716af707f2027e1.rlib --extern regex=/Users/kali/dev/snips/dinghy/target/debug/deps/libregex-36c8e259ac5ba542.rlib --extern clap=/Users/kali/dev/snips/dinghy/target/debug/deps/libclap-c293040e3ef9c3f9.rlib --extern log=/Users/kali/dev/snips/dinghy/target/debug/deps/liblog-bf16bb9a4912b11d.rlib --extern error_chain=/Users/kali/dev/snips/dinghy/target/debug/deps/liberror_chain-554f08ea2cb4f0f5.rlib --extern dinghy=/Users/kali/dev/snips/dinghy/target/debug/deps/libdinghy.rlib -L framework=/System/Library/PrivateFrameworks -L native=/Users/kali/dev/snips/dinghy/target/debug/build/libgit2-sys-e36168b6cc5d8ac9/out/lib -L native=/usr/local/Cellar/libssh2/1.8.0/lib -L native=/usr/local/opt/openssl/lib -L native=/Users/kali/dev/snips/dinghy/target/debug/build/openssl-5464f8f6e728c35a/out -L native=/Users/kali/dev/snips/dinghy/target/debug/build/openssl-sys-extras-5c7e4d8925825f00/out -L native=/Users/kali/dev/snips/dinghy/target/debug/build/miniz-sys-60c8d67696f63a43/out`

Let's check out your and see if we see something significantly different.

[learnopengles]

Hi kali,

I think it might have nothing to do with dinghy as I'm currently working on isolating it, and with most of dinghy gone it still happens.

Looking at your cargo output, mine is also compiling the same versions, though there are some differences in the command line after accounting for path names: https://www.diffchecker.com/l5FjV3kx

[kali]

yeah. The line about libgit2-sys is actually significantly weird. On my side it's linking against a libssh2-sys that has been build by the create (i think) while yours use one from brew. I do have libssh2 installed from brew, it's "libssh2: stable 1.8.0 (bottled), HEAD".

[kali]

the switch could come from here... https://github.com/alexcrichton/git2-rs/blob/master/libgit2-sys/build.rs#L34

[learnopengles]

I wonder if that's cause of openssl; I had to install it with macports as that crate fails to compile for me otherwise. hmm...

---

No luck with that env var. I'll keep digging and let you know. :)

[learnopengles]

So after uninstalling macports and all ports and installing openssl through brew, it's working! I'll close the issue since it doesn't seem like an issue with Dinghy, and try to figure out why it wasn't working with MacPorts.

[kali]

Good! have fun discovering Rust, and feel free to reopen if needed.

[learnopengles]

So here is the solution:

Assuming: sudo port install openssl export OPENSSL_INCLUDE_DIR=/opt/local/include export OPENSSL_LIB_DIR=/opt/local/lib

Then this will compile dinghy: CMAKE_PREFIX_PATH=/opt/local/ cargo install dinghy

The problem is that the libgit2 CMake script is picking up iconv from /usr/lib while Cargo is picking it up from /opt/local/lib. Passing that flag in the env var forces CMake to pick it up from the same place as Cargo.

A better solution might be for them to both pick it up from /usr/lib, but right now I'm not sure how to do that. At least this seems to work, too.

[kali]

hey! wanna try the simulator support i just pushed ?

[learnopengles]

It works! I suppose you didn't push it to crates.io yet? ;)

[learnopengles]

Is all of the AMDevice* stuff needed only when running on the device? It's used for finding out the right parameters to pass to LLDB? I'm starting to really appreciate what you said in email about hacks...

[learnopengles]

Hmm, I noticed that xcrun simctl launch... doesn't return a non-zero error code when the test fails.

[kali]

yeah the AMDevice stuff is for finding the devices on USB (or bluetooth, but i have not decided if i want to support that yet). It also the way in to get lldb running on the device, which is apparently the only way to run an app remotely (that's what ios-deploy and everybody else is doing : they reverse engineered this private apple api, I just wrote rust ffi bindings as needed).

About simctl launch, I had not noticed and, well, this is not good news. It probably mean I'll have to go through some more lldb hacking.

Thanks for trying, I'll keep you posted.

[learnopengles]

Ah OK I see. Argh... I'm used to hacks on Android but I was hoping the iOS side would be more free of them. This seems worse. :S

So for simctl launch there are two ways you could do it. With the LLVM route it's similar to what you're already doing except you won't need to use any AMDevice stuff:

1) Get path of app on simulator:

xcrun simctl get_app_container booted com.rust.tests

2) Connect using LLDB:

lldb
platform select ios-simulator
platform connect <device-id>
target create /Users/<user>/Library/Developer/CoreSimulator/Devices/<device-id>/data/Containers/Bundle/Application/<app-id>/app.app/hello
run

I did run into an issue with this where a second run in the same LLVM doesn't show stdout.

The second way of doing this kind of cheating, but it works, too: just read the output and check for "test result: FAILED" and return the appropriate code. ;)

[kali]

Waw, thanks for figuring this out. I'll try to get this running ASAP.

I don't want to "cheat" if possible by using knowledge of whatever we are running. I would like to market dinghy as agnostic as possible (the cargo-dinghy part being really what I just need right now :) ).

I also wonder which bit in the process is killing the colors.

[learnopengles]

Me too! I noticed that too.

If it helps please feel free to take anything you need from here: https://gist.github.com/learnopengles/ebf6c04180ca7d81b83e37945b1870cc

I adapted it from your code + some other articles anyways. :)

Right now I'm trying to figure out how to integrate all of this into Rust's own unit tests so we can have the compiler itself run the tests on the simulator.

[kali]

Hey, I gave a shot at implementing the lldb command you were mentioning but run into a problem:

(lldb) platform select ios-simulator
  Platform: ios-simulator
    Triple: x86_64h-apple-macosx
OS Version: 10.12.2 (16C67)
    Kernel: Darwin Kernel Version 16.3.0: Thu Nov 17 20:23:58 PST 2016; root:xnu-3789.31.2~1/RELEASE_X86_64
  Hostname: 127.0.0.1
WorkingDir: /Users/kali/dev/snips/dinghy
  SDK Path: error: unable to locate SDK
No devices are available.
(lldb) platform connect XXXXX-XXXX-XXX-XXx-XXXXXX
error: no device with UDID or name 'XXXXX-XXXX-XXX-XXx-XXXXXX' was found

Do you get the same SDK Path error at the first step ? If not where is it finding the SDK ? I might be missing some obscure bit of XCode...

[kali]

all right, just need to use xcrun.

[learnopengles]

You needed to do xcrun lldb? Hmm interesting... that wasn't necessary for me. Everything else in that output looks the same for me.

[learnopengles]

I think will be better to run it through lldb as I noticed using xcrun simctl launch ... has some odd behavior occasionally on Travis: the binary gets run twice. I don't know what causes that as I haven't seen it happen locally, but possibly it won't happen if the binary is executed through lldb.

The main downside of lldb I've seen is that stdout is only shown the first time, so lldb would have to be closed and restarted if the test was to be restarted. I guess this might not be an issue in practice.

[kali]

ok, got the simulator running. Note that the device code was also ignoring the return value, but it is now fixed. Tell me how it works for you if you want. I'll probably push an updated crate later today or tomorrow.

[learnopengles]

Hi Mathieu, sorry, I missed the second part of your message. I'll try it out now.

[learnopengles]

It works great! Returns 1 when it fails and 0 otherwise.

[beefon]

@kali how did you make lldb work with simulators? I type xcrun lldb and platform select ios-simulator, but I always get No devices are available:

(lldb) platform select ios-simulator
  Platform: ios-simulator
    Triple: x86_64h-apple-macosx
OS Version: 10.15.3 (19D76)
    Kernel: Darwin Kernel Version 19.3.0: Thu Jan  9 20:58:23 PST 2020; root:xnu-6153.81.5~1/RELEASE_X86_64
  Hostname: 127.0.0.1
WorkingDir: /Users/USER/
  SDK Path: "/Applications/Xcode_11_3.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator.sdk"
No devices are available.

It correctly detects SDK path, but platform connect <UDID> fails with error: no device with UDID or name '<UDID>' was found

[simlay]

@beefon, it’s an issue with llvm as mentioned here: https://github.com/snipsco/dinghy/issues/41#issuecomment-573767640