Got 'unexpected cargo internal error' with 1.44.0-nightly while building some project

[Blaxar]

Problem

I'm having trouble building some project with cargo because it fails to fetch a specific package from sources (with a given version/tag).

While I don't see why this is necessarily cargo's fault rather than the project's fault, the command output is kindly suggesting me to fill a bug report, so here I am.

Steps

1. Use cargo 1.44.0-nightly (390e8f245 2020-04-07), got it from updating with rustup.
2. Clone the following project (and checkout a specific commit, just to be sure):

   $ git clone git@gitlab.freedesktop.org:gstreamer/gst-plugins-rs.git
   $ cd gst-plugins-rs && git checkout bd90d1d9

   [EDIT] 2-bis. Project page if you can't clone it ssh-style: https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs [/EDIT]

3. Run cargo build --verbose inside the resulting project folder.

This is the output I get:

$ cargo build --verbose
error: failed to download `gio-sys v0.7.0 (https://github.com/gtk-rs/sys#508d9264)`

Caused by:
  unable to get packages from source

Caused by:
  failed to find gio-sys v0.7.0 (https://github.com/gtk-rs/sys#508d9264) in path source
note: this is an unexpected cargo internal error
note: we would appreciate a bug report: https://github.com/rust-lang/cargo/issues/
note: cargo 1.44.0-nightly (390e8f245 2020-04-07)

Possible Solution(s)

• Have an actual verbose output in this case when using the --verbose flag: you get the same amount of info without using said flag.
• If this is not an actual cargo internal error and just a project-side error, then do not display it as such.

Notes

Output of cargo version: cargo 1.44.0-nightly (390e8f245 2020-04-07) Output of rustc --version: rustc 1.44.0-nightly (3712e11a8 2020-04-12) OS being used: ArchLinux Release channel: Nightly

[alexcrichton]

Thanks for the report! It definitely looks like this is a bug on Cargo's end classifying this as an internal error, but I can't seem to clone your repository to try to investigate further. Mind gisting Cargo.toml and maybe Cargo.lock too?

[Blaxar]

Sure thing!

I don't know if the root Cargo.toml file will be that useful on its own, but here you go: https://gist.github.com/Blaxar/8f65e91123ed82f65b44e18397ab64b5

Also, here's the page for said project if you want to get a better look: https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs

You might be able to clone this via https using git clone https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs.git.

[alexcrichton]

Thanks! I can't seem to reproduce, though, mind gisting your Cargo.lock as well?

[Blaxar]

Cargo.lock is already in the gisted content that I linked :)

As for you not being able to reproduce this: that's weird, but in theory this error may show up if you specify an invalid version tag for a listed dependency, I guess ?

I'll look a bit more into it on my side, but I'm not well-versed in cargo or rustc debugging, sadly.

[alexcrichton]

Ok trying to minimize this I got:

# Cargo.toml
[package]
name = "foo"
version = "0.1.0"
authors = ["Alex Crichton <alex@alexcrichton.com>"]
edition = "2018"

[dependencies]
gio-sys = { git = "https://github.com/gtk-rs/sys" }

and

# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
[[package]]
name = "foo"
version = "0.1.0"
dependencies = [
 "gio-sys",
]

[[package]]
name = "gio-sys"
version = "0.7.0"
source = "git+https://github.com/gtk-rs/sys#508d9264d6f82f63fdb7c4b87cc9834ce2e80404"
dependencies = [
 "glib-sys",
 "gobject-sys",
 "libc",
 "pkg-config",
]

[[package]]
name = "glib-sys"
version = "0.9.1"
source = "git+https://github.com/gtk-rs/sys#6bd0cab694a0b446d5fe530cd87bf2b499f065ff"
dependencies = [
 "libc",
 "pkg-config",
]

[[package]]
name = "gobject-sys"
version = "0.9.1"
source = "git+https://github.com/gtk-rs/sys#6bd0cab694a0b446d5fe530cd87bf2b499f065ff"
dependencies = [
 "glib-sys",
 "libc",
 "pkg-config",
]

[[package]]
name = "libc"
version = "0.2.68"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dea0c0405123bba743ee3f91f49b1c7cfb684eef0da0a50110f758ccf24cdff0"

[[package]]
name = "pkg-config"
version = "0.3.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "05da548ad6865900e60eaba7f589cc0783590a92e940c26953ff81ddbab2d677"

executing cargo fetch on that workspace yields the internal error here.

It may be a bit more reducable but the thing that I noticed is that there's two git revisions in this lock file for the same repository, which in theory is an invalid lock file. @Blaxar did this lock file perhaps get generated from a git-merge? Presumably if you run cargo update locally it works too, right?

(just trying to figure out more context for how this came about myself, it's still a bug we need to fix and this is likely actionable enough)

[Blaxar]

Well I just ran cargo update and it basically fixed everything on my end, cargo fetch also works after that, so thanks for this!

As to how I ended up here: that's a good question, especially since there's no Cargo.lock file being versioned on this repository (and there shouldn't be), so I don't see how a merge could have done this when pulling from remote.

I basically tried to check this project after a while (it built just fine last time) so I pulled-in the latest changes from master and then things went south.

[ehuss]

@alexcrichton I'm wondering if this might be indirectly related to #7841. I've been researching that issue off and on. There is something fishy about the way git sources are hashed and stored in the SourceMap. Because the git rev is not included in the hash, if you have multiple revs of the same git source they stomp on each other when building the SourceMap. I'm wondering if there might be other situations where if you have multiple revs of the same git repo that the lack of precise comparison causes problems.

[alexcrichton]

@ehuss I suspect it's the same issue yeah. My guess is that git merges will produce a "corrupt" lock file which has two git sources that shouldn't actually exist. Turns out this was actually made worse by the recent change to Cargo.lock format!

I think we'll need to fix this at lockfile-loading time to either unlock the git dependency if we find two locked versions or something similar. The way that the git sha is handled in hashing and such is really nuanced and likely too clever for its own good. It's somewhat intentional I believe that these two sources hash to the same value. Within the resolver this is expected behavior, but when consuming external input it produces bugs like this, so we'll need to sanitize it on its way to the resolver.