holmgr / cargo-sweep

A cargo subcommand for cleaning up unused build files generated by Cargo
MIT License
693 stars 31 forks source link

Doesn't clean non-workspace crates that exist inside a workspace #66

Closed bes closed 1 year ago

bes commented 2 years ago

I have a git with the following layout

.
├── Cargo.toml
├── app-wasm
│  ├── Cargo.toml
├── cli
│  ├── Cargo.toml
├── crates
│  ├── api
│  │  ├── Cargo.toml
│  ├── models
│  │  ├── Cargo.toml
...

Where the top-level Cargo.toml looks like:

[workspace]
members = [
    "crates/*",
]
exclude = [
    "cli",
    "app-wasm",
]

Where cli and app-wasm are standalone applications that use the workspace crates.

I used the following commands:

Actual:

Expected:

I am willing to work on a PR if this can be confirmed to be a bug.

Thanks!

jyn514 commented 1 year ago

I can't reproduce this bug. I have your workspace layout and ran

; cargo run --manifest-path cli/Cargo.toml 
   Compiling cli v0.1.0 (/home/jnelson/rust-community/cargo-sweep-66/cli)
    Finished dev [unoptimized + debuginfo] target(s) in 0.21s
     Running `cli/target/debug/cli`
; cargo sweep -r --maxsize 0
[INFO] Successfully removed: "/home/jnelson/rust-community/cargo-sweep-66/cli/target/debug/deps/cli-a05f7f2d31159c21"
[INFO] Successfully removed: "/home/jnelson/rust-community/cargo-sweep-66/cli/target/debug/deps/cli-a05f7f2d31159c21.d"
[INFO] Successfully removed: "/home/jnelson/rust-community/cargo-sweep-66/cli/target/debug/.fingerprint/cli-a05f7f2d31159c21"
[INFO] Cleaned 3.96 MiB

Can you post your version of cargo-sweep?

bes commented 1 year ago

Sure, I just reproduced it --

cargo install-update -l
Package           Installed           Latest              Needs update
cargo-sweep       v0.6.2              v0.6.2              No

Then I run

# Contains lots of top-level projects, some of which are rust projects,
# some of which are workspace projects,
# some of which have folders with projects in them.
cargo sweep -r --maxsize 0 "$HOME/repos/"
[INFO] Cleaned 0.00 bytes
[INFO] Cleaned 0.00 bytes
[INFO] Cleaned 0.00 bytes
[INFO] Cleaned 0.00 bytes
[INFO] Cleaned 0.00 bytes
[INFO] Cleaned 0.00 bytes
[INFO] Cleaned 0.00 bytes
[INFO] Cleaned 0.00 bytes
[INFO] Cleaned 0.00 bytes
[INFO] Cleaned 0.00 bytes
[INFO] Cleaned 0.00 bytes
[INFO] Cleaned 0.00 bytes
[INFO] Cleaned 0.00 bytes
[INFO] Cleaned 0.00 bytes
[INFO] Cleaned 0.00 bytes
[INFO] Cleaned 0.00 bytes
[INFO] Cleaned 0.00 bytes
[INFO] Cleaned 0.00 bytes
[INFO] Cleaned 0.00 bytes
[INFO] Cleaned 0.00 bytes
[INFO] Cleaned 0.00 bytes
[INFO] Cleaned 0.00 bytes
[INFO] Cleaned 0.00 bytes
[INFO] Cleaned 0.00 bytes
[INFO] Cleaned 0.00 bytes
[INFO] Cleaned 0.00 bytes
[INFO] Cleaned 0.00 bytes
[INFO] Cleaned 0.00 bytes

Now check one of the subprojects...

cd repos/company-name/workspace-project-folder/non-workspace-subfolder/target
du -h .

 16K    ./release/.fingerprint/crossbeam-epoch-8ae397b9da37dc89
 16K    ./release/.fingerprint/swift-bridge-ir-6c74206b15dbb725
8.0K    ./release/.fingerprint/quote-6a7327282a4672f5
8.0K    ./release/.fingerprint/proc-macro2-2979496c850aadf5
 16K    ./release/.fingerprint/pebble-ios-f62ef605c6f6f0fa
 16K    ./release/.fingerprint/tokio-macros-9a06342b42913104
 16K    ./release/.fingerprint/quote-75ed8aa68ebecbfb
 16K    ./release/.fingerprint/async-trait-4afa65846246732f
 16K    ./release/.fingerprint/libc-fb4c6b2baee25cad
 16K    ./release/.fingerprint/serde_derive-4a156cacdcfd2adb
... A lot more
 16K    ./aarch64-apple-ios/release/.fingerprint/signal-hook-registry-6cee91521f014614
 16K    ./aarch64-apple-ios/release/.fingerprint/untrusted-31ac7c520c8fb430
 16K    ./aarch64-apple-ios/release/.fingerprint/idna-48c003e0e9fb658f
 16K    ./aarch64-apple-ios/release/.fingerprint/webpki-aa82e8c44dacbbcf
 16K    ./aarch64-apple-ios/release/.fingerprint/tracing-core-dfb8653bf2c04219
 16K    ./aarch64-apple-ios/release/.fingerprint/socket2-8f2869cfa2cc49c4
 16K    ./aarch64-apple-ios/release/.fingerprint/http-8635f79f5448518f
 16K    ./aarch64-apple-ios/release/.fingerprint/mime-76aa990a8fa6102d
 16K    ./aarch64-apple-ios/release/.fingerprint/bytes-59a50d53babe16da
... A lot more
 14M    ./aarch64-apple-ios-sim/debug/incremental/private-symbol-here
 37M    ./aarch64-apple-ios-sim/debug/incremental/private-symbol-here
 37M    ./aarch64-apple-ios-sim/debug/incremental/private-symbol-here
5.5M    ./aarch64-apple-ios-sim/debug/incremental/private-symbol-here
5.5M    ./aarch64-apple-ios-sim/debug/incremental/private-symbol-here
 85M    ./aarch64-apple-ios-sim/debug/incremental/private-symbol-here
 85M    ./aarch64-apple-ios-sim/debug/incremental/private-symbol-here
9.6M    ./aarch64-apple-ios-sim/debug/incremental/private-symbol-here
... A lot more
  0B    ./debug/build/encoding_rs-1747592e3b4b3c5d/out
8.0K    ./debug/build/encoding_rs-1747592e3b4b3c5d
1.7M    ./debug/build/crc32fast-12828e40be895ae5
  0B    ./debug/build/parking_lot_core-3d6e85baf63bab8a/out
 12K    ./debug/build/parking_lot_core-3d6e85baf63bab8a
1.6M    ./debug/build/memoffset-035911e99805014d
1.8M    ./debug/build/libc-e968c670dd108265
4.0K    ./debug/build/anyhow-cd4743d14c29a12f/out

 15G    .

So cargo-sweep, in this instance, failed to remove 15 GiB of data in the target folder for me.

bes commented 1 year ago

And if I do it from the "worspace project folder" I get this

cd repos/company-name/workspace-project-folder/non-workspace-subfolder
cargo sweep -r --maxsize 0
[INFO] Cleaned 0.00 bytes
bes commented 1 year ago

I'm happy to provide any information you think is valuable.

bes commented 1 year ago

If I run cargo-sweep from the directory itself, it works

cd repos/company-name/workspace-project-folder/non-workspace-subfolder
cargo sweep -r --maxsize 0
... Lots of files deleted
[INFO] Cleaned 10.66 GiB
bes commented 1 year ago

Note: I have multiple non-workspace subfolders, i.e. multiple non-workspace Cargo.toml's and projects.

bes commented 1 year ago

I had some time over, so I looked into it. It's pretty clear to me that this never worked, and I don't know how you got it to work @jyn514 .

main.rs:

            if let Some(target_directory) = is_cargo_root(entry.path()) {
                target_paths.insert(target_directory);
                iter.skip_current_dir(); // no reason to look at the src and such
            }

As soon as it finds a directory with a valid Cargo root, it just skips all subdirectories.

bes commented 1 year ago

If I just comment out the iter.skip_current_dir(); line, it magically starts working.

jyn514 commented 1 year ago

Ah ok, the problem is multiple root workspaces, not that the directory is ignored altogether. Seems like a simple fix - do you want to make a PR with that change? :)

bes commented 1 year ago

Yes, I was making a patch already <3

bes commented 1 year ago

Thanks @jyn514 🎉

PSeitz commented 1 year ago

Nice, can we have a new release for that?