Closed bazhenov closed 7 months ago
I have the same issue here under Linux, but with version 0.83. I'm almost thinking this problem has another source, because the last weeks the build process was running normal.
Can it be, that it is related to the cc create, or another? Because when I revert my Cargo.lock file to an older state it works. But when I run cargo update
, the build process hangs again.
Edit: Yes I'm almost sure is the cc-rs crate, when you revert to 1.0.79 is works.
Indeed! It works for 1.0.79.
For anyone who faces this issue. The following snippet of code in Cargo.toml
is required to override the dependency temporarily
[patch.crates-io]
cc = { git = "https://github.com/rust-lang/cc-rs.git", version = "1.0", tag = "1.0.79" }
Then run cargo update
, only then cargo build
.
The problem was introduced in cc
crate version 1.0.80 and seems to be resolved in 1.0.81 (see: https://github.com/rust-lang/cc-rs/issues/841 and https://github.com/rust-lang/cc-rs/pull/842). But it still doesn't work for me. 1.0.79 is the latest version that is working for me right now.
Thanks for the investigation! I’ll try to update the Cargo.toml to exclude the affected versions
@bazhenov, are you sure that 1.0.80 introduced this bug? I had the intention that the bug come up with 1.0.81.
We should also create a report.
@bazhenov, are you sure that 1.0.80 introduced this bug? I had the intention that the bug come up with 1.0.81.
Hm... I'm sure it works in my environment on 1.0.79 and doesn't work on 1.0.80 and 1.0.81. So it seems like so, but I will not bet on it at this point 😀
We should also create a report.
I mention this issue in the https://github.com/rust-lang/cc-rs/issues/841 because it seems like they are related. I will create separate report if it will turn out as an independent issue.
Only change in cc v1.0.81 is to accept non-utf8 output from compiler, this problem doesn't seem like a utf-8 problem (which should cause panic), so I'd say probably both v1.0.{80, 81} is affected and would better report it as a new issue.
@bazhenov From your stack trace, I can see that cc::Build::compile_objects
is blocked waiting for tokens from jobserver.
This should not happen in cc
since it waits for compiler process to be done in a separate thread to avoid deadlock, so the only thing I can conclude is that somehow your jobserver has all its token exhausted.
I can reproduce the problem locally on cc
version 1.0.80 and 1.0.81, 1.0.79 works fine for me. I have just released the version 0.84.4 of the crate that restricts cc
dependency to <=1.0.79.
Ok, I realize there could be one bug in cc::Build::compile_objects that caused this:
Wait-thread could exit early due to error, however the spawn-thread will go on and keep spawning, thus eventually hitting a deadlock.
Would open a PR shortly.
I looks like, that opencv is now in conflict with the ring crate. In my project I wanted to update jsonwebtoken to version 9, but it depends on ring version v0.17.4, and that wants cc v1.0.83.
Any idea how to fix that?
Full error message is:
error: failed to select a version for `cc`.
... required by package `opencv v0.85.0`
... which satisfies dependency `opencv = "^0.85"` of package `manager-api v0.10.0 (/home/jb/dev/gitea/manager-api)`
versions that meet the requirements `<=1.0.79` are: 1.0.79, 1.0.78, 1.0.77, 1.0.76, 1.0.75, 1.0.74, 1.0.73, 1.0.72, 1.0.71, 1.0.70, 1.0.69, 1.0.68, 1.0.67, 1.0.66, 1.0.65, 1.0.64, 1.0.63, 1.0.62, 1.0.61, 1.0.60, 1.0.59, 1.0.58, 1.0.57, 1.0.56, 1.0.55, 1.0.54, 1.0.53, 1.0.52, 1.0.51, 1.0.50, 1.0.49, 1.0.48, 1.0.47, 1.0.46, 1.0.45, 1.0.44, 1.0.43, 1.0.42, 1.0.41, 1.0.40, 1.0.39, 1.0.38, 1.0.37, 1.0.36, 1.0.35, 1.0.34, 1.0.33, 1.0.32, 1.0.31, 1.0.30, 1.0.29, 1.0.28, 1.0.27, 1.0.26, 1.0.25, 1.0.24, 1.0.23, 1.0.22, 1.0.21, 1.0.20, 1.0.19, 1.0.18, 1.0.17, 1.0.16, 1.0.15, 1.0.14, 1.0.13, 1.0.12, 1.0.11, 1.0.10, 1.0.9, 1.0.8, 1.0.7, 1.0.6, 1.0.5, 1.0.4, 1.0.3, 1.0.2, 1.0.1, 1.0.0, 0.0.1
the package `opencv` depends on `cc`, with features: `parallel` but `cc` does not have these features.
all possible versions conflict with previously selected packages.
previously selected package `cc v1.0.83`
... which satisfies dependency `cc = "^1.0.83"` of package `ring v0.17.4`
... which satisfies dependency `ring = "^0.17.4"` of package `jsonwebtoken v9.0.0`
... which satisfies dependency `jsonwebtoken = "^9"` of package `manager-api v0.10.0 (/home/jb/dev/gitea/manager-api)`
failed to select a version for `cc` which could resolve this conflict
Can you try running cargo update
to see if it fixes the conflict?
No , if I run that, I get immediately the same error. I also run cargo clean and rm -rf ~/.cargo/registry ~/.cargo/.package-cache
, to be sure there is no old conflicts.
When I downgrade jsonwebtoken to version 8 again, it works again.
Well, there are some problems with dependencies:
cc
as is because newer versions depend on jobserver
and 2 crates can't use jobserver
's Client::from_env()
at the same time because it hangs on the second call (unless https://github.com/rust-lang/jobserver-rs/pull/57 is merged and released)jobslot
to fix the hang issue, but even with this, the newest version of jobserver
(0.1.27) uses some new API calls that were stabilized after rust 1.59 (which is the MSRV of opencv
crate) without specifying msrv
in Cargo.toml
and that results in the broken build on older rust versions. This has been fixed in https://github.com/rust-lang/jobserver-rs/pull/59, but it's not released yet.So at this moment I can't bump cc
without breaking something else, so let's wait until either of those 2 problems is fixed.
Yes, I understand. I can stick on jsonwebtoken v8 for now.
P.S. I'm working on https://github.com/rust-lang/cc-rs/pull/889 which completely removes jobserver as a dependency of cc by using vendored implementation and it also removes thread creation in parallel compile_objects
(2 less thread spawned during compilation if feature cc/parallel is enabled).
Bad news is that cc now has marv of 1.53 https://github.com/rust-lang/cc-rs/pull/889#issuecomment-1788887737
but it seems that this crate has a msrv of 1.59 so not affected
Hi @NobodyXu, it looks like there is an issue with contested resources in the new version of cc
. I can see a lot of "Resource temporary unavailable" in the Linux CI jobs: https://github.com/twistedfall/opencv-rust/actions/runs/6844842064/job/18609212991?pr=511
Hi @twistedfall , are you still using jobserver along with cc?
If that's the case, it's probably just the old bug hitting back https://github.com/rust-lang/jobserver-rs/pull/57 .
Since cc's own vendored jobserver implementation now sets the jobserver fd as non-blocking, with the bug it might have set stderr as non-blocking accidentally.
However, there's a workaround.
If you call cc's try_compile
/compile
before creating a jobserver in your code, then you will be fine since cc will dup
the jobserver fd.
Any compile
with more than one object will do the job.
Thanks, I'll try to employ this workaround!
version = "<=1.0.79"
is causing some build issues for me.
Any luck with the workaround?
Well, the version 1.0.84 that the workaround was applicable to has since been yanked (because of https://github.com/rust-lang/cc-rs/issues/902) and no new version has been released so situation hasn't really changed.
@torsteingrindvik Can you please check the NobodyXu-use-jobslot
branch to see if it fixes your build and doesn't introduce any new issues?
@torsteingrindvik Can you please check the
NobodyXu-use-jobslot
branch to see if it fixes your build and doesn't introduce any new issues?
This branch works for me (i.e. cargo dependencies resolve now and no compile issues) and a brief test didn't show any problems.
This should now be fixed in v0.88.8 (at the cost of bumping MSRV from 1.65 to 1.66)
brew install opencv
cargo build
command hangs. Seems like the build scripts is waiting output of thecc
compiler:But I see no compiler process running on behalf of build script:
Maybe compiler failed with some error, but it's not displayed on the screen.