JuliaPackaging / BinaryBuilderBase.jl

https://juliapackaging.github.io/BinaryBuilderBase.jl/stable
MIT License
11 stars 31 forks source link

Add `manage_shards` to interactively remove compiler shards #303

Closed sloede closed 1 year ago

sloede commented 1 year ago

This PR is inspired by #206 (and, if merged, supersedes it) and based on its code by @Crghilardi. It adds a manage_shards function that allows one to interactively view the currently installed compiler shards and then remove them. This can be helpful if one experiments with a lot of BB builds and ends up with tens of GB of compiled shards that clutter up the Julia depot.

Default usage (without deleting anything):

julia> using BinaryBuilderBase

julia> manage_shards()
Which compiler shards should be removed (total size on disk: 79.666 GiB)?
[press: d=done, a=all, n=none]
 > [ ] (4.062 GiB) CompilerShard("GCCBootstrap", v"8.1.0", macOS x86_64 {os_version=14.0.0}, Linux x86_64 {libc=musl}, :unpacked)
   [ ] (3.179 GiB) CompilerShard("GCCBootstrap", v"4.8.5", macOS x86_64 {os_version=14.0.0}, Linux x86_64 {libc=musl}, :unpacked)
   [ ] (2.649 GiB) CompilerShard("GCCBootstrap", v"10.2.0", Linux aarch64 {libc=glibc}, Linux x86_64 {libc=musl}, :unpacked)
   [ ] (2.635 GiB) CompilerShard("GCCBootstrap", v"10.2.0", Linux x86_64 {libc=glibc}, Linux x86_64 {libc=musl}, :unpacked)
   [ ] (2.479 GiB) CompilerShard("GCCBootstrap", v"10.2.0", Linux x86_64 {libc=musl}, Linux x86_64 {libc=musl}, :unpacked)
   [ ] (2.470 GiB) CompilerShard("GCCBootstrap", v"10.2.0", Linux i686 {libc=glibc}, Linux x86_64 {libc=musl}, :unpacked)
   [ ] (2.426 GiB) CompilerShard("GCCBootstrap", v"10.2.0", FreeBSD x86_64 {os_version=12.2.0}, Linux x86_64 {libc=musl}, :unpacked)
   [ ] (2.414 GiB) CompilerShard("GCCBootstrap", v"10.2.0", Linux aarch64 {libc=musl}, Linux x86_64 {libc=musl}, :unpacked)
   [ ] (2.359 GiB) CompilerShard("GCCBootstrap", v"10.2.0", Linux i686 {libc=musl}, Linux x86_64 {libc=musl}, :unpacked)
v  [ ] (2.307 GiB) CompilerShard("GCCBootstrap", v"10.2.0", Linux armv7l {call_abi=eabihf, libc=glibc}, Linux x86_64 {libc=musl}, :unpacked)
Removed 0 compiler shards.

Sorted by size on disk and deleting 2 shards:

julia> manage_shards(sort_by=:size, rev=true)
Which compiler shards should be removed (total size on disk: 79.666 GiB)?
[press: d=done, a=all, n=none]
   [X] (4.609 GiB) BinaryBuilderBase.CompilerShard("GCCBootstrap", v"10.2.0", macOS x86_64 {os_version=14.0.0}, Linux x86_64 {libc=musl}, :unpacked)
   [ ] (4.062 GiB) BinaryBuilderBase.CompilerShard("GCCBootstrap", v"8.1.0", macOS x86_64 {os_version=14.0.0}, Linux x86_64 {libc=musl}, :unpacked)
   [ ] (3.179 GiB) BinaryBuilderBase.CompilerShard("GCCBootstrap", v"4.8.5", macOS x86_64 {os_version=14.0.0}, Linux x86_64 {libc=musl}, :unpacked)
 > [X] (2.649 GiB) BinaryBuilderBase.CompilerShard("GCCBootstrap", v"10.2.0", Linux aarch64 {libc=glibc}, Linux x86_64 {libc=musl}, :unpacked)
   [ ] (2.635 GiB) BinaryBuilderBase.CompilerShard("GCCBootstrap", v"10.2.0", Linux x86_64 {libc=glibc}, Linux x86_64 {libc=musl}, :unpacked)
   [ ] (2.481 GiB) BinaryBuilderBase.CompilerShard("GCCBootstrap", v"10.2.0", Linux powerpc64le {libc=glibc}, Linux x86_64 {libc=musl}, :unpacked)
   [ ] (2.479 GiB) BinaryBuilderBase.CompilerShard("GCCBootstrap", v"10.2.0", Linux x86_64 {libc=musl}, Linux x86_64 {libc=musl}, :unpacked)
   [ ] (2.470 GiB) BinaryBuilderBase.CompilerShard("GCCBootstrap", v"10.2.0", Linux i686 {libc=glibc}, Linux x86_64 {libc=musl}, :unpacked)
   [ ] (2.426 GiB) BinaryBuilderBase.CompilerShard("GCCBootstrap", v"10.2.0", FreeBSD x86_64 {os_version=12.2.0}, Linux x86_64 {libc=musl}, :unpacked)
v  [ ] (2.414 GiB) BinaryBuilderBase.CompilerShard("GCCBootstrap", v"10.2.0", Linux aarch64 {libc=musl}, Linux x86_64 {libc=musl}, :unpacked)
The following shards have been marked for removal:

  (4.609 GiB) BinaryBuilderBase.CompilerShard("GCCBootstrap", v"10.2.0", macOS x86_64 {os_version=14.0.0}, Linux x86_64 {libc=musl}, :unpacked)
  (2.649 GiB) BinaryBuilderBase.CompilerShard("GCCBootstrap", v"10.2.0", Linux aarch64 {libc=glibc}, Linux x86_64 {libc=musl}, :unpacked)

Are you sure that these should be removed? [Y/n]: y
Removed 2 compiler shards.

As stated by Casey, it is unclear if something like this is even wanted in BBB, but I found it hugely useful for myself, and I thought I'd share it here.