tikv / jemallocator

Rust allocator using jemalloc as a backend
Other
347 stars 58 forks source link

Unnecessary build files left in the `target` directory #89

Open Kobzol opened 4 months ago

Kobzol commented 4 months ago

When jemalloc-sys builds the jemalloc native dependency, it first creates a build directory on disk, then runs make and make install, which installs a copy of the built libs into a lib directory, amongst other things. However, after the build finishes, the build directory is still kept on disk. It takes ~220 MiB on my x64 Linux, which is quite a lot! Especially considering that it can exist in multiple copies (debug/release, multiple repos using jemalloc etc.).

I think that the build.rs script should delete the build directory from disk once the compilation (make install) successfully finishes, since it shouldn't be needed anymore, and I don't think that Cargo will ever reuse it (if something relevant changes, Cargo will compile jemalloc from scratch in a different directory, AFAIK).

What do you think?

BusyJay commented 4 months ago

Interesting, but deleting a directory can also be time consuming and contributed to build time, which I think most people may notice and avoid. If the directory is too large, you can always run cargo clean.

Kobzol commented 4 months ago

Interesting, but deleting a directory can also be time consuming and contributed to build time, which I think most people may notice and avoid.

I doubt that deleting a directory will cause any large bottlenecks during compilation. I did a small benchmark where I simply run cargo build --release with/without deleting the directory, using hyperfine:

Original code:

Benchmark 1: cargo build --release
Time (mean ± σ):     20.091 s ±  0.953 s    [User: 141.451 s, System: 13.891 s]
Range (min … max):   19.373 s … 21.818 s    10 runs

Modified code (that removes the build directory):

Benchmark 1: cargo build --release
Time (mean ± σ):     19.493 s ±  0.115 s    [User: 138.811 s, System: 13.807 s]
Range (min … max):   19.317 s … 19.680 s    10 runs

The results are pretty much the same. Granted, I have an SSD disk, but even then, deleting a directory really shouldn't be a big issue in the grand scheme of Rust compilation. You can find my patched version here.

Also, the build script currently does make install, but it never uses the installed libraries in <OUT_DIR>/lib. So IMO it should either delete the build directory and use the installed libraries, or just don't do make install, when the results are unused.

If the directory is too large, you can always run cargo clean.

I can, but the problem is that it removes everything. I have a bunch of projects using jemalloc, and I work on them regularly. The problem is that once I do the initial build, the build directory isn't useful anymore, but it sits on disk and consumes ~200 MiB x debug/release/other profile x the amount of projects using jemalloc :) That seems wasteful.

Kobzol commented 1 month ago

@BusyJay So, what do you think? :) I can send a PR if you'd like.