Improve compile times

[Wumpf]

(edited by @emilk)

Improve build times, and aspire to reduce number of dependencies and code bloat in the process.

Some useful tools include:

• cargo clean && cargo build --timings -p rerun
• cargo tree
• cargo machete
• https://github.com/google/bloaty
• https://github.com/RazrFalcon/cargo-bloat
• https://github.com/dtolnay/cargo-llvm-lines
• cargo udeps to find unused dependencies

See also:

• https://fasterthanli.me/articles/why-is-my-rust-build-so-slow

Reducing dependencies

Some ideas:

• replace clap with something simpler, like pico-args
• replace sha2 with a simpler hash
• replace chrono with time
• replace image crate with https://github.com/etemesi254/zune-image
• re_web_server should be able to use something must simpler than hyper+tokio
• replace lazy_static with once_cell ?
• remove strum
• enumset can be replaced with something simpler (maybe by macro_rules!)
• reduce naga features?
• crossbeam (std::sync::mpsc is now using the same implementation) - though crossbeam compiles in 2s, so probably not worth it

Done:

• https://github.com/rerun-io/rerun/pull/1425
• https://github.com/rerun-io/rerun/pull/1407
• https://github.com/rerun-io/rerun/pull/1406

Other tasks

• 347

• Track build time and number of dependencies over time like we do for benchmarks
• Add cargo-deny to the CI of other crates, like arboard, to prevent duplicate dependencies of e.g. nix
• In puffin, add support for lz4_flex: https://github.com/EmbarkStudios/puffin/pull/130#issuecomment-1495641210

Other

We have some instructions for improving build times in https://github.com/rerun-io/rerun/blob/main/BUILD.md#improving-compile-times

[Wumpf]

rebuild on dev with M1 max via cargo clean && cargo build --timings

cargo-timing-20230215T130337Z.html.zip

Quite surprised about zlib being so slow here! Investigating alternatives!

[Wumpf]

very similar results on second run

[Wumpf]

Getting rid of zstd is somewhat hard - it is just really really good it seems https://github.com/EmbarkStudios/puffin/pull/130

[emilk]

I think we can switch from zstd to lz4_flex. To get the benefits we must first remove zstd from our puffin dependency:

• https://github.com/EmbarkStudios/puffin/issues/131

EDIT:

• https://github.com/EmbarkStudios/puffin/pull/135
• https://github.com/rerun-io/rerun/pull/2112

[emilk]

Current state on my M1 MacBook Pro

sccache --stop-server && rm -rf /Users/emilk/Library/Caches/Mozilla.sccache && sccache --start-server
cargo clean && cargo build --timings -p rerun

2m 12s

[emilk]

This halves compilation times for users only using the rerun crate as a logging library:

• https://github.com/rerun-io/rerun/pull/2183

[emilk]

Things has improved - cargo clean && cargo build --timings -p rerun on my M1 MacBook Pro:

78.8s and 353 dependencies

[Wumpf]

Status update: On c5f817686c6a1b8723ddeee25a9aed0ff4fa1e92 (somewhere between 0.16 and 0.17 ;)

M1max MacBook Pro (very similar to emils machine above) 66.5s and 336 dependencies

re_types is the biggest unit with quite some distance, otherwise the next logical thing would be to try to get rid of serde (should be in reach for the sdk!)