search

JuliaPackaging / BinaryBuilderBase.jl

https://juliapackaging.github.io/BinaryBuilderBase.jl/stable
MIT License
11 stars 31 forks source link

[CI] Do not run full tests when using unpacked shards #325

Closed giordano closed 10 months ago

giordano commented 11 months ago

Alternative to #321. Not a great solution, but at least we can get some tests to actually run.

Close #321.

giordano commented 11 months ago

So the problem wasn't disk space after all :confused:

gbaraldi commented 11 months ago

I wonder if it's just squashfs= false that's broken. But we only test it in 1.7?

Keno commented 11 months ago

Can someone try running that config locally to see if it passes?

giordano commented 10 months ago

Can someone try running that config locally to see if it passes?

Yesterday I ran the tests with

export BINARYBUILDER_FULL_SHARD_TEST=true
export BINARYBUILDER_RUNNER=unprivileged
export BINARYBUILDER_USE_SQUASHFS=false

and the full suite was successful for me. I could see all the shards being downloaded one by one, no one failed to be installed.

giordano commented 10 months ago

I'll merge this PR after CI finishes. I marked a failing test as broken (issue tracked by #347), there are some failing tests when using Clang, but as far as I understand they'll be fixed by #345 (and it's quite interesting this bug wasn't triggered by the compilers in the squashfs images :confused:). After that PR, we should actually have green CI also for the no-squashfs runners. Not running the full compiler suite isn't ideal, but it takes long time and we can't cache those shards, so another benefit of this PR is shorter CI time, too. If I'll ever have more time for long debugging sessions, I'll try to further track down the culprit in the full shard suite in #346.

giordano commented 10 months ago

Ok, I further looked into #346: when using the full shard testsuite with julia v1.9 we get

  Unable to automatically download/install artifact 'GCCBootstrap-x86_64-linux-gnu.v9.1.0.x86_64-linux-musl.unpacked' from sources listed in '/home/runner/work/BinaryBuilderBase.jl/BinaryBuilderBase.jl/Artifacts.toml'.
  Sources attempted:
  - https://pkg.julialang.org/artifact/7e2927c9465a3b39a76029dd203cb225fc98fe1a
      Error: AssertionError: size == padded_size == 0
  - https://github.com/JuliaPackaging/Yggdrasil/releases/download/GCCBootstrap-v9.1.0+3/GCCBootstrap-x86_64-linux-gnu.v9.1.0.x86_64-linux-musl.unpacked.tar.gz
      Error: AssertionError: size == padded_size == 0

This error message typically indicates that there's indeed not enough space on disk to unpack the tarball (see https://github.com/JuliaIO/Tar.jl/pull/161), and the tests fail shortly after that point (there's another compiler shard being downloaded, maybe that one somehow manages to download before the failure). However, if running only the offending testsuite, tests are successful, which suggests there isn't anything broken with the specific compiler shard being downloaded, and reinforces the hypothesis that the failure is only due to insufficient space on disk.

Thus, given the resources we're currently using I don't think we can do anything else apart from reducing the amount of compiler shards being downloaded when running the tests with the unpackaged images.