libbpf / libbpf-sys

Rust bindings to libbpf from the Linux kernel
https://crates.io/crates/libbpf-sys/
BSD 2-Clause "Simplified" License
200 stars 48 forks source link

elfutils: use https protocol for submodule #77

Closed anakryiko closed 8 months ago

anakryiko commented 8 months ago

Using git:// protocol for submodules quite often causes issues in corporate setting where development machines are behind firewalls. For http/https usually some transparent proxy would be provided, while git protocol isn't proxied as nicely.

Let's avoid unnecessary complications and use https:// protocol for elfutils submodule.

alexforster commented 8 months ago

Fine by me.

NB: We have a few changes pending on master now, but I want to wait until the next version of libbpf is released before cutting a new libbpf-sys, if that's okay with everyone.

anakryiko commented 8 months ago

Yeah, I've been asked for a new libbpf release due to some of the pending changes, so we might have a new one pretty soon.

danielocfb commented 8 months ago

There may be an incompatibility caused by this change where when we build libbpf-rs and cargo attempts to fetch submodules. It seemingly fails to do so now:

(libbpf-rs)/: cargo build
    Updating git repository `https://github.com/libbpf/libbpf-sys.git`
    Updating git submodule `https://sourceware.org/git/elfutils.git`
warning: spurious network error (3 tries remaining): early EOF; class=Net (12); code=Eof (-20)
warning: spurious network error (2 tries remaining): early EOF; class=Net (12); code=Eof (-20)
warning: spurious network error (1 tries remaining): early EOF; class=Net (12); code=Eof (-20)
error: failed to load source for dependency `libbpf-sys`

Caused by:
  Unable to update https://github.com/libbpf/libbpf-sys.git?rev=9ed58f7dd737496396b6ea052e771539b3c76e8c

Caused by:
  failed to update submodule `elfutils`

Caused by:
  failed to fetch submodule `elfutils` from https://sourceware.org/git/elfutils.git

Caused by:
  network failure seems to have happened
  if a proxy or similar is necessary `net.git-fetch-with-cli` may help here
  https://doc.rust-lang.org/cargo/reference/config.html#netgit-fetch-with-cli

Caused by:
  early EOF; class=Net (12); code=Eof (-20)

If I set CARGO_NET_GIT_FETCH_WITH_CLI everything works, but I suspect it's a bit problematic if we can't build with the default settings. I don't think there is an additional proxy involved right now and if I pin to an earlier libbpf-sys revision everything goes through fine. So this looks as if https://sourceware.org/git/elfutils.git may rely on git features not present in Cargo's git support. Haven't investigated further.

alexforster commented 8 months ago

On my work laptop I'm forced to use a MitM'ing proxy, and we have to set git-fetch-with-cli to true all over the place. Is there something similar happening here?

danielocfb commented 8 months ago

On my work laptop I'm forced to use a MitM'ing proxy, and we have to set git-fetch-with-cli to true all over the place. Is there something similar happening here?

I suspect the reason to be different (it's not proxy related, but looks more like a limitation of libgit which is used by cargo), but yeah, that would certainly be an easy way out. It would mean users would need to have git installed, but at this point I suppose that's considered a given.

danielocfb commented 8 months ago

So I tried that setting in .cargo/config.toml, but it seems as if just adding it to this project may not be sufficient. Rather, it would need to be in effect in the consuming project (e.g., libbpf-rs in our case). So that makes it a no-go from my perspective.