search

libgit2 / libgit2sharp

Git + .NET = ❤
http://libgit2.github.com
MIT License
3.21k stars 890 forks source link

System.DllNotFoundException: lib/linux/x86_64/libgit2-1196807.so on Ubuntu Linux #1585

Open evil-shrike opened 6 years ago

evil-shrike commented 6 years ago

I have difficulties with LibGit2Sharp while running Cake (with Cake.Git addin) build on Linux Ubuntu 16.04 in Docker - https://github.com/cake-contrib/Cake_Git/issues/68 But I managed to isolate issue with LibGit2Sharp only.

Reproduction steps

touch gt.cs

using LibGit2Sharp;
using System;

public class Program
{
    public static void Main()
    {
        Console.WriteLine("HI");
        var repo = new Repository("/opt/buildagent/work/e1fb3d71588668ec");
    }
}

csc gt.cs /r:LibGit2Sharp.dll

mono gt.exe


HI

Unhandled Exception: System.TypeInitializationException: The type initializer for 'LibGit2Sharp.Core.NativeMethods' threw an exception. ---> System.DllNotFoundException: lib/linux/x86_64/libgit2-1196807.so at (wrapper managed-to-native) LibGit2Sharp.Core.NativeMethods.git_libgit2_init() at LibGit2Sharp.Core.NativeMethods+LibraryLifetimeObject..ctor () [0x00006] in :0 at LibGit2Sharp.Core.NativeMethods..cctor () [0x00054] in :0 --- End of inner exception stack trace --- at LibGit2Sharp.Core.Proxy.git_repository_open (System.String path) [0x00008] in :0 at LibGit2Sharp.Repository..ctor (System.String path, LibGit2Sharp.RepositoryOptions options, LibGit2Sharp.Repository+RepositoryRequiredParameter requiredParameter) [0x00312] in :0 at LibGit2Sharp.Repository..ctor (System.String path) [0x00000] in :0 at Program.Main () [0x0000c] in :0 [ERROR] FATAL UNHANDLED EXCEPTION: System.TypeInitializationException: The type initializer for 'LibGit2Sharp.Core.NativeMethods' threw an exception. ---> System.DllNotFoundException: lib/linux/x86_64/libgit2-1196807.so at (wrapper managed-to-native) LibGit2Sharp.Core.NativeMethods.git_libgit2_init() at LibGit2Sharp.Core.NativeMethods+LibraryLifetimeObject..ctor () [0x00006] in :0 at LibGit2Sharp.Core.NativeMethods..cctor () [0x00054] in :0 --- End of inner exception stack trace --- at LibGit2Sharp.Core.Proxy.git_repository_open (System.String path) [0x00008] in :0 at LibGit2Sharp.Repository..ctor (System.String path, LibGit2Sharp.RepositoryOptions options, LibGit2Sharp.Repository+RepositoryRequiredParameter requiredParameter) [0x00312] in :0 at LibGit2Sharp.Repository..ctor (System.String path) [0x00000] in :0 at Program.Main () [0x0000c] in :0


Mono looks up for native module in subfolder bacause of config
>cat LibGit2Sharp.dll.config
```xml
<configuration>
    <dllmap os="linux" cpu="x86-64" wordsize="64" dll="git2-1196807" target="lib/linux/x86_64/libgit2-1196807.so" />
    <dllmap os="osx" cpu="x86,x86-64" dll="git2-1196807" target="lib/osx/libgit2-1196807.dylib" />
</configuration>

the file lib/linux/x86_64/libgit2-1196807.so does exist. also tried to copy it besides my .exe module.

Expected behavior

no error

Actual behavior

System.TypeInitializationException: The type initializer for 'LibGit2Sharp.Core.NativeMethods' threw an exception. ---> System.DllNotFoundException: lib/linux/x86_64/libgit2-1196807.so

Version of LibGit2Sharp (release number or SHA1)

libgit2-1196807.so

Operating system(s) tested; .NET runtime tested

# cat /etc/os-release
NAME="Ubuntu"
VERSION="16.04.3 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.3 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial
# mono --version
Mono JIT compiler version 5.12.0.226 (tarball Thu May  3 09:48:32 UTC 2018)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
        TLS:           __thread
        SIGSEGV:       altstack
        Notifications: epoll
        Architecture:  amd64
        Disabled:      none
        Misc:          softdebug
        Interpreter:   yes
        LLVM:          supported, not enabled.
        GC:            sgen (concurrent by default)
evil-shrike commented 6 years ago

As a workaround for me worked this trick with installing libgit2-24 globally and setup its target path in LibGit2Sharp.dll.config (as /usr/lib/x86_64-linux-gnu/libgit2.so.24). But as I'm using it via Cake/Cake.Git it's very hard to use this workaround.

evil-shrike commented 6 years ago

I seems I solved the issue by installing apt-get isntall libc6 libcurl3. Not sure about the need of libc6 but after I installed libcurl3 libgit2sharp began working.

That comment helps - https://github.com/libgit2/libgit2sharp/issues/1472#issuecomment-315584303

bording commented 6 years ago

@evil-shrike curl is a requirement, so it sounds like you've fixed the problem!

evil-shrike commented 6 years ago

@bording yeah, but may be it makes sense to put the required curl module besides libgit2-1196807.so? Or at least put some warning in Readme?

bording commented 6 years ago

Attempting to bundle other libraries instead of linking against the system versions is not something we want to do, especially for something like OpenSSL.

Adding something to the README that lists all of the required native dependencies is a good idea!

ethomson commented 6 years ago

@bording yeah, but may be it makes sense to put the required curl module besides libgit2-1196807.so? Or at least put some warning in Readme?

We are not going to ship OpenSSL (which we would have to do, transitively), and you don’t want us to. If there are security holes in it, you want a single source of OpenSSL that you can patch. You want to know that you fixed all of them when you do. You don’t want some version that we installed still being vulnerable.

pauldotknopf commented 6 years ago

I agree about openssl, but I think you should statically link libcurl. It is a problematic dependency that various greatly between Ubuntu versions and packages on Ubuntu.

I've had more trouble with libcurl than any other library, trying to find the "sweet spot" of configuration for my OS and docker images.

https://github.com/getinsomnia/insomnia/issues/878

pauldotknopf commented 6 years ago

Also, the latest .NET Core 2.1.1 depends upon libcurl4.

System.Net.Http.Native.so:
    linux-vdso.so.1 (0x00007ffd677ef000)
    libcurl.so.4 => /usr/lib/x86_64-linux-gnu/libcurl.so.4 (0x00007f77ccfaa000)
    libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f77ccc1c000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f77cc87e000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f77cc666000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f77cc275000)
    libnghttp2.so.14 => /usr/lib/x86_64-linux-gnu/libnghttp2.so.14 (0x00007f77cc050000)
    libidn2.so.0 => /usr/lib/x86_64-linux-gnu/libidn2.so.0 (0x00007f77cbe33000)
    librtmp.so.1 => /usr/lib/x86_64-linux-gnu/librtmp.so.1 (0x00007f77cbc17000)
    libpsl.so.5 => /usr/lib/x86_64-linux-gnu/libpsl.so.5 (0x00007f77cba09000)
    libssl.so.1.1 => /usr/lib/x86_64-linux-gnu/libssl.so.1.1 (0x00007f77cb79f000)
    libcrypto.so.1.1 => /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1 (0x00007f77cb327000)
    libgssapi_krb5.so.2 => /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2 (0x00007f77cb0dc000)
    libldap_r-2.4.so.2 => /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2 (0x00007f77cae8a000)
    liblber-2.4.so.2 => /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2 (0x00007f77cac7c000)
    libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f77caa5f000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f77ca840000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f77cd42d000)
    libunistring.so.2 => /usr/lib/x86_64-linux-gnu/libunistring.so.2 (0x00007f77ca4c2000)
    libgnutls.so.30 => /usr/lib/x86_64-linux-gnu/libgnutls.so.30 (0x00007f77ca15d000)
    libhogweed.so.4 => /usr/lib/x86_64-linux-gnu/libhogweed.so.4 (0x00007f77c9f29000)
    libnettle.so.6 => /usr/lib/x86_64-linux-gnu/libnettle.so.6 (0x00007f77c9cf3000)
    libgmp.so.10 => /usr/lib/x86_64-linux-gnu/libgmp.so.10 (0x00007f77c9a72000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f77c986e000)
    libkrb5.so.3 => /usr/lib/x86_64-linux-gnu/libkrb5.so.3 (0x00007f77c9598000)
    libk5crypto.so.3 => /usr/lib/x86_64-linux-gnu/libk5crypto.so.3 (0x00007f77c9366000)
    libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007f77c9162000)
    libkrb5support.so.0 => /usr/lib/x86_64-linux-gnu/libkrb5support.so.0 (0x00007f77c8f57000)
    libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007f77c8d3c000)
    libsasl2.so.2 => /usr/lib/x86_64-linux-gnu/libsasl2.so.2 (0x00007f77c8b21000)
    libgssapi.so.3 => /usr/lib/x86_64-linux-gnu/libgssapi.so.3 (0x00007f77c88e0000)
    libp11-kit.so.0 => /usr/lib/x86_64-linux-gnu/libp11-kit.so.0 (0x00007f77c85b1000)
    libtasn1.so.6 => /usr/lib/x86_64-linux-gnu/libtasn1.so.6 (0x00007f77c839e000)
    libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1 (0x00007f77c819a000)
    libheimntlm.so.0 => /usr/lib/x86_64-linux-gnu/libheimntlm.so.0 (0x00007f77c7f91000)
    libkrb5.so.26 => /usr/lib/x86_64-linux-gnu/libkrb5.so.26 (0x00007f77c7d04000)
    libasn1.so.8 => /usr/lib/x86_64-linux-gnu/libasn1.so.8 (0x00007f77c7a62000)
    libhcrypto.so.4 => /usr/lib/x86_64-linux-gnu/libhcrypto.so.4 (0x00007f77c782c000)
    libroken.so.18 => /usr/lib/x86_64-linux-gnu/libroken.so.18 (0x00007f77c7616000)
    libffi.so.6 => /usr/lib/x86_64-linux-gnu/libffi.so.6 (0x00007f77c740e000)
    libwind.so.0 => /usr/lib/x86_64-linux-gnu/libwind.so.0 (0x00007f77c71e5000)
    libheimbase.so.1 => /usr/lib/x86_64-linux-gnu/libheimbase.so.1 (0x00007f77c6fd6000)
    libhx509.so.5 => /usr/lib/x86_64-linux-gnu/libhx509.so.5 (0x00007f77c6d8c000)
    libsqlite3.so.0 => /usr/lib/x86_64-linux-gnu/libsqlite3.so.0 (0x00007f77c6a83000)
    libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007f77c684b000)
libgit2-b0d9952.so: /usr/lib/x86_64-linux-gnu/libcurl.so.4: version `CURL_OPENSSL_3' not found (required by /docgen/runtimes/linux-x64/native/libgit2-b0d9952.so)
        linux-vdso.so.1 (0x00007ffdbbaa4000)
        librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007ff922d2b000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007ff922b0c000)
        libcurl.so.4 => /usr/lib/x86_64-linux-gnu/libcurl.so.4 (0x00007ff92288d000)
        libssl.so.1.0.0 => /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 (0x00007ff922625000)
        libcrypto.so.1.0.0 => /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0 (0x00007ff9221e2000)
        libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007ff921fc5000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007ff921bd4000)
        /lib64/ld-linux-x86-64.so.2 (0x00007ff923244000)
        libnghttp2.so.14 => /usr/lib/x86_64-linux-gnu/libnghttp2.so.14 (0x00007ff9219af000)
        libidn2.so.0 => /usr/lib/x86_64-linux-gnu/libidn2.so.0 (0x00007ff921792000)
        librtmp.so.1 => /usr/lib/x86_64-linux-gnu/librtmp.so.1 (0x00007ff921576000)
        libpsl.so.5 => /usr/lib/x86_64-linux-gnu/libpsl.so.5 (0x00007ff921368000)
        libssl.so.1.1 => /usr/lib/x86_64-linux-gnu/libssl.so.1.1 (0x00007ff9210fe000)
        libcrypto.so.1.1 => /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1 (0x00007ff920c86000)
        libgssapi_krb5.so.2 => /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2 (0x00007ff920a3b000)
        libldap_r-2.4.so.2 => /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2 (0x00007ff9207e9000)
        liblber-2.4.so.2 => /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2 (0x00007ff9205db000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007ff9203d7000)
        libunistring.so.2 => /usr/lib/x86_64-linux-gnu/libunistring.so.2 (0x00007ff920059000)
        libgnutls.so.30 => /usr/lib/x86_64-linux-gnu/libgnutls.so.30 (0x00007ff91fcf4000)
        libhogweed.so.4 => /usr/lib/x86_64-linux-gnu/libhogweed.so.4 (0x00007ff91fac0000)
        libnettle.so.6 => /usr/lib/x86_64-linux-gnu/libnettle.so.6 (0x00007ff91f88a000)
        libgmp.so.10 => /usr/lib/x86_64-linux-gnu/libgmp.so.10 (0x00007ff91f609000)
        libkrb5.so.3 => /usr/lib/x86_64-linux-gnu/libkrb5.so.3 (0x00007ff91f333000)
        libk5crypto.so.3 => /usr/lib/x86_64-linux-gnu/libk5crypto.so.3 (0x00007ff91f101000)
        libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007ff91eefd000)
        libkrb5support.so.0 => /usr/lib/x86_64-linux-gnu/libkrb5support.so.0 (0x00007ff91ecf2000)
        libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007ff91ead7000)
        libsasl2.so.2 => /usr/lib/x86_64-linux-gnu/libsasl2.so.2 (0x00007ff91e8bc000)
        libgssapi.so.3 => /usr/lib/x86_64-linux-gnu/libgssapi.so.3 (0x00007ff91e67b000)
        libp11-kit.so.0 => /usr/lib/x86_64-linux-gnu/libp11-kit.so.0 (0x00007ff91e34c000)
        libtasn1.so.6 => /usr/lib/x86_64-linux-gnu/libtasn1.so.6 (0x00007ff91e139000)
        libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1 (0x00007ff91df35000)
        libheimntlm.so.0 => /usr/lib/x86_64-linux-gnu/libheimntlm.so.0 (0x00007ff91dd2c000)
        libkrb5.so.26 => /usr/lib/x86_64-linux-gnu/libkrb5.so.26 (0x00007ff91da9f000)
        libasn1.so.8 => /usr/lib/x86_64-linux-gnu/libasn1.so.8 (0x00007ff91d7fd000)
        libhcrypto.so.4 => /usr/lib/x86_64-linux-gnu/libhcrypto.so.4 (0x00007ff91d5c7000)
        libroken.so.18 => /usr/lib/x86_64-linux-gnu/libroken.so.18 (0x00007ff91d3b1000)
        libffi.so.6 => /usr/lib/x86_64-linux-gnu/libffi.so.6 (0x00007ff91d1a9000)
        libwind.so.0 => /usr/lib/x86_64-linux-gnu/libwind.so.0 (0x00007ff91cf80000)
        libheimbase.so.1 => /usr/lib/x86_64-linux-gnu/libheimbase.so.1 (0x00007ff91cd71000)
        libhx509.so.5 => /usr/lib/x86_64-linux-gnu/libhx509.so.5 (0x00007ff91cb27000)
        libsqlite3.so.0 => /usr/lib/x86_64-linux-gnu/libsqlite3.so.0 (0x00007ff91c81e000)
        libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007ff91c5e6000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007ff91c248000)

As it stands, I don't think LibGit2Sharp.NativeBinaries currently work on the latest .NET Core.

Try to use it in the latest microsoft/dotnet:2.1.1-runtime-bionic docker image.

pauldotknopf commented 6 years ago

I rebuilt libgit2 using the latest .NET Core docker images.

diff --git a/Dockerfile.linux-x64 b/Dockerfile.linux-x64
index 3a5c69c..745a352 100644
--- a/Dockerfile.linux-x64
+++ b/Dockerfile.linux-x64
@@ -1,4 +1,4 @@
-FROM ubuntu:14.04
+FROM microsoft/dotnet:2.1.1-runtime-bionic
 WORKDIR /nativebinaries
 COPY . /nativebinaries/

This works. It is a lot more setup with my docker images, considering I know how to manually build libgit2 and put it in the right place, but it works now.

pauldotknopf commented 6 years ago

I am building libgit2 as a part of my docker build now. This ensures the resulting binary is valid for the platform.

FROM microsoft/dotnet:2.1.1-runtime-bionic
RUN apt-get update
RUN apt -y install cmake libcurl4-openssl-dev libssl-dev pkg-config git
RUN apt-get clean

RUN git clone https://github.com/libgit2/libgit2.git /libgit2 && \
    cd /libgit2 && \
    git checkout b0d9952c318a3d1b8917e06ad46b9110c0c28831

RUN cd /libgit2 && \
    cmake -DCMAKE_BUILD_TYPE:STRING=Release \
      -DBUILD_CLAR:BOOL=OFF \
      -DUSE_SSH=OFF \
      -DENABLE_TRACE=ON \
      -DLIBGIT2_FILENAME=git2-b0d9952 \
      -DCMAKE_OSX_ARCHITECTURES="i386;x86_64" \
      . && \
      cmake --build .

FROM microsoft/dotnet:2.1.1-runtime-bionic
COPY --from=0 /libgit2/libgit2-b0d9952.so /yourapp/runtimes/linux-x64/native
bording commented 6 years ago

We are currently using the same native library for all versions of Ubuntu. If 18.04 has changed the version of curl it ships, then we'll have to build a specific version of the library for it and ship it in the native binaries package.

tzachshabtay commented 6 years ago

This was reproduced on Debian 9 as well, so not just an Ubuntu issue.

bording commented 6 years ago

The latest 0.26.0 preview release includes both an Ubuntu 18.04 and Debian 9 binary, so it should work.

magne commented 5 years ago

The following seems to work just fine on Ubuntu 18.10. I've tested with GitVersion modified to use these versions.

LibGit2Sharp 0.26.0-preview-0070
LigGit2Sharp.NativeBinaries 1.0.258
bording commented 5 years ago

@magne Do you have libssl1.0.0 installed via APT? I would expect you'd have to before 18.10 would work.

magne commented 5 years ago

Yes, libssl1.0.0 (and libssl1.1) is installed.

bording commented 5 years ago

My understanding is that libssl1.0.0 is not installed by default on 18.10. Is that something you installed?

magne commented 5 years ago

As you can see when I try to remove libssl1.0.0, it's dependents include the .Net Core runtimes and the openssh client. These I have installed. But it also includes the snap daemon, and I belive that was present after the initial OS installation.

> sudo apt remove libssl1.0.0
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
  dotnet-host dotnet-hostfxr-2.1 dotnet-hostfxr-2.2 liblttng-ust-ctl4 liblttng-ust0 liburcu6 ncurses-term
Use 'sudo apt autoremove' to remove them.
The following packages will be REMOVED:
  aspnetcore-runtime-2.1 aspnetcore-runtime-2.2 dotnet-runtime-2.1 dotnet-runtime-2.2 dotnet-runtime-deps-2.1 dotnet-runtime-deps-2.2 dotnet-sdk-2.1 dotnet-sdk-2.2
  gnome-software-plugin-snap libssl1.0.0 openssh-client openssh-server openssh-sftp-server snapd ssh-import-id
0 upgraded, 0 newly installed, 15 to remove and 0 not upgraded.

I've also tried to spin up the ubuntu-minimal:bionic and ubuntu-minimal:cosmic LXC images, and both of those include libssl1.0.0 by default.

bording commented 5 years ago

Thanks for the additional information!

That doesn't seem to line up with my initial testing, so it looks like I've got some more investigation to do.

However, if we can reasonably assume that libssl1.0.0 is installed on 18.04 & 18.10, then that definitely simplifies things.

AdmiringWorm commented 5 years ago

I got hit with this same problem of libgit2sharp not being able to load the native library (In my case, I had all dependencies needed for the library, but running on Arch Linux instead of Ubuntu. Which is probably not officially supported).

I was able to solve this by adding a new dllmap in the LibGit2Sharp.dll.config config file to fallback to the system installed libgit2.so library.

I appended the following to the configuration file (without removing anything)

<dllmap os="linux" dll="git2-6311e88" target="libgit2.so" />

Hopefully this helps someone else coming across this issue with the same problem.

PulsarFX commented 4 years ago

On debian 9 with v0.26.0 and libgit2sharp.nativebinaries.2.0.278 I needed to change from

<dllmap os="linux" cpu="x86-64" wordsize="64" dll="git2-572e4d8" target="lib/linux-x64/libgit2-572e4d8.so" />
to
<dllmap os="linux" cpu="x86-64" wordsize="64" dll="git2-572e4d8" target="lib/debian.9-x64/libgit2-572e4d8.so" />
nohwnd commented 4 years ago

@maulwuff thanks, that helped me work around the issue (on WSL2 Ubuntu 20.04 .NETCore 3.1.401)