search

RazrFalcon / cargo-bloat

Find out what takes most of the space in your executable.
MIT License
2.31k stars 50 forks source link

Cargo bloaty reports only a tiny portion of the whole binary #83

Closed fzyzcjy closed 2 years ago

fzyzcjy commented 2 years ago

Hi thanks for the lib! However, Cargo bloaty reports only a tiny portion of the whole binary.

My .so file for android is about 10MB (not zipped, stripped). It contains some pure Rust code, especially using ndarray crate so I guess that take up some space. It also links to OpenCV (a C++ library) using the opencv crate, so I guess that also takes up some space.

However, when I run cargo bloaty it only reports <1MB:

(base) ➜  rust git:(master) cargo ndk --target armeabi-v7a -- bloat --release --crates
[2021-11-24T03:31:37Z INFO  cargo_ndk::cli] Using NDK at path: /Users/tom/Library/Android/sdk/ndk/21.2.6472646
[2021-11-24T03:31:37Z INFO  cargo_ndk::cli] NDK API level: 21
[2021-11-24T03:31:37Z INFO  cargo_ndk::cli] Building targets: armeabi-v7a
[2021-11-24T03:31:37Z INFO  cargo_ndk::cli] Building armeabi-v7a (armv7-linux-androideabi)
    Finished release [optimized + debuginfo] target(s) in 1.62s
    Analyzing target/armv7-linux-androideabi/release/vision_utils_rs

File  .text     Size Crate
0.1%  45.9% 115.4KiB [Unknown]
0.0%  29.8%  74.8KiB std
0.0%   7.1%  17.9KiB rustc_demangle
0.0%   6.7%  16.9KiB gimli
0.0%   4.3%  10.8KiB addr2line
0.0%   2.9%   7.4KiB miniz_oxide
0.0%   2.4%   6.0KiB opencv
0.0%   0.2%     602B adler
0.0%   0.2%     420B indexmap
0.0%   0.0%      50B object
0.0%   0.0%      48B vision_utils_rs
0.0%   0.0%      12B protobuf
0.1% 100.0% 251.3KiB .text section size, the file size is 167.8MiB

Note: numbers above are a result of guesswork. They are not 100% correct and never will be.

Here is another attempt:

(base) ➜  rust git:(master) cargo ndk --target armeabi-v7a -- bloat --release 
[2021-11-24T03:30:27Z INFO  cargo_ndk::cli] Using NDK at path: /Users/tom/Library/Android/sdk/ndk/21.2.6472646
[2021-11-24T03:30:27Z INFO  cargo_ndk::cli] NDK API level: 21
[2021-11-24T03:30:27Z INFO  cargo_ndk::cli] Building targets: armeabi-v7a
[2021-11-24T03:30:27Z INFO  cargo_ndk::cli] Building armeabi-v7a (armv7-linux-androideabi)
    Finished release [optimized + debuginfo] target(s) in 1.23s
    Analyzing target/armv7-linux-androideabi/release/vision_utils_rs

File  .text     Size          Crate Name
0.0%   4.6%  11.6KiB            std std::backtrace_rs::symbolize::gimli::elf::<impl std::backtrace_rs::symbolize::gimli::Mapping>::new::{...
0.0%   3.8%   9.5KiB            std std::backtrace_rs::symbolize::gimli::resolve
0.0%   2.7%   6.7KiB      addr2line addr2line::ResUnit<R>::parse_lines
0.0%   2.3%   5.7KiB    miniz_oxide miniz_oxide::inflate::core::decompress
0.0%   1.7%   4.3KiB          gimli gimli::read::unit::parse_attribute
0.0%   1.5%   3.7KiB          gimli gimli::read::rnglists::RngListIter<R>::next
0.0%   1.2%   3.0KiB rustc_demangle rustc_demangle::v0::Printer::print_type
0.0%   1.0%   2.5KiB rustc_demangle rustc_demangle::demangle
0.0%   1.0%   2.5KiB            std core::slice::sort::recurse
0.0%   0.9%   2.2KiB rustc_demangle <rustc_demangle::legacy::Demangle as core::fmt::Display>::fmt
0.0%   0.8%   2.1KiB      addr2line addr2line::Function<R>::parse_children
0.0%   0.8%   2.1KiB          gimli gimli::read::abbrev::Abbreviations::insert
0.0%   0.8%   2.0KiB          gimli gimli::read::unit::Attribute<R>::value
0.0%   0.8%   1.9KiB          gimli gimli::read::line::parse_attribute
0.0%   0.7%   1.8KiB rustc_demangle rustc_demangle::v0::Printer::print_path
0.0%   0.6%   1.5KiB      [Unknown] __itt_init_ittlib
0.0%   0.6%   1.4KiB      [Unknown] _ZN2cvL14initInterTab2DEib
0.0%   0.6%   1.4KiB rustc_demangle rustc_demangle::v0::Printer::print_const
0.0%   0.6%   1.4KiB rustc_demangle rustc_demangle::v0::Parser::skip_path
0.0%   0.6%   1.4KiB      [Unknown] android_cpuInit
0.1%  72.2% 181.6KiB                And 1546 smaller methods. Use -n N to show more.
0.1% 100.0% 251.3KiB                .text section size, the file size is 167.8MiB

The text section should not be as small as 0.25MB. Indeed, when I run (naive) bloaty on the resulting .so file (that is to be packaged into android app), it says about 5MB:

bloaty ~/temp/hello.so
    FILE SIZE        VM SIZE    
 --------------  -------------- 
  55.4%  5.56Mi  50.3%  5.56Mi    .text
  32.0%  3.22Mi  29.1%  3.22Mi    .rodata
   0.0%       0   9.1%  1.00Mi    .bss
   4.0%   406Ki   3.6%   406Ki    .ARM.extab
   3.3%   337Ki   3.0%   337Ki    .data.rel.ro
   3.3%   337Ki   3.0%   337Ki    .rel.dyn
   1.6%   160Ki   1.4%   160Ki    .ARM.exidx
   0.2%  23.3Ki   0.2%  23.2Ki    .data
   0.1%  7.75Ki   0.1%  7.71Ki    .dynstr
   0.1%  7.30Ki   0.1%  7.27Ki    .dynsym
   0.0%  4.51Ki   0.0%  4.47Ki    .got
   0.0%  3.50Ki   0.0%  3.46Ki    .plt
   0.0%  2.89Ki   0.0%  2.85Ki    .hash
   0.0%  2.34Ki   0.0%  2.30Ki    .rel.plt
   0.0%  2.20Ki   0.0%       0    [Unmapped]
   0.0%  1.25Ki   0.0%     492    [11 Others]
   0.0%     970   0.0%     930    .gnu.version
   0.0%     585   0.0%       0    .comment
   0.0%     484   0.0%     444    .gnu.hash
   0.0%     344   0.0%     304    .dynamic
   0.0%     330   0.0%     330    [LOAD #1 [RX]]
 100.0%  10.0Mi 100.0%  11.0Mi    TOTAL

More info:

bloaty ~/temp/hello.so --debug-file /Users/tom/QAPMain/Development/frontend/vision_utils/rust/target/armv7-linux-androideabi/release/libvision_utils_rs.so -d compileunits
    FILE SIZE        VM SIZE    
 --------------  -------------- 
  29.0%  2.92Mi  26.4%  2.92Mi    [section .rodata]
  18.3%  1.84Mi  16.7%  1.84Mi    [423 Others]
  11.1%  1.11Mi  10.1%  1.11Mi    [section .text]
   0.0%       0   9.0%  1020Ki    [section .bss]
   8.5%   871Ki   7.7%   871Ki    src/lib.rs
   5.4%   554Ki   4.9%   554Ki    /Users/tom/.cargo/registry/src/github.com-1ecc6299db9ec823/reqwest-0.11.5/src/lib.rs
   4.0%   406Ki   3.6%   406Ki    [section .ARM.extab]
   3.3%   337Ki   3.0%   337Ki    [section .rel.dyn]
   3.1%   320Ki   2.8%   321Ki    /Users/tom/.cargo/registry/src/github.com-1ecc6299db9ec823/protobuf-2.25.1/src/lib.rs
   2.4%   251Ki   2.2%   251Ki    [section .data.rel.ro]
   1.9%   193Ki   1.7%   193Ki    /Users/tom/.cargo/registry/src/github.com-1ecc6299db9ec823/rustls-0.19.1/src/lib.rs
   1.8%   180Ki   1.6%   180Ki    /Users/tom/.cargo/git/checkouts/sentry-rust-92aa47acd509410f/4f12673/sentry/src/lib.rs
   1.6%   163Ki   1.4%   163Ki    /Users/tom/.cargo/registry/src/github.com-1ecc6299db9ec823/h2-0.3.6/src/lib.rs
   1.6%   160Ki   1.4%   160Ki    [section .ARM.exidx]
   1.5%   149Ki   1.3%   149Ki    /Users/tom/.cargo/registry/src/github.com-1ecc6299db9ec823/encoding_rs-0.8.28/src/lib.rs
   1.4%   147Ki   1.3%   147Ki    /Users/tom/.cargo/registry/src/github.com-1ecc6299db9ec823/regex-1.5.4/src/lib.rs
   1.2%   125Ki   1.1%   125Ki    /Users/tom/.cargo/registry/src/github.com-1ecc6299db9ec823/regex-syntax-0.6.25/src/lib.rs
   1.1%   108Ki   1.0%   108Ki    /Users/tom/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.12.0/src/lib.rs
   1.0%   106Ki   0.9%   106Ki    /Users/tom/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.13/src/lib.rs
   1.0%   103Ki   0.9%   103Ki    library/std/src/lib.rs
   0.9%  97.0Ki   0.9%  97.0Ki    /build/master_pack-android/opencv/3rdparty/libtiff/tif_fax3sm.c
 100.0%  10.0Mi 100.0%  11.0Mi    TOTAL

Could you please give some hint on where I should do? Thanks!

RazrFalcon commented 2 years ago

You're saying that your library is about 10MB, but cargo bloat reports 167.8MiB. Are you sure you want to analyze vision_utils_rs and not other library?

fzyzcjy commented 2 years ago

@RazrFalcon Hi thanks for the quick reply!

167MB is because it contains debug symbol (because bloaty does not work without debug symbols IMHO). But for end users they get a 10MB .so file without debug symbols

RazrFalcon commented 2 years ago

end users they get a 10MB .so file

But cargo bloat analyzes vision_utils_rs, not vision_utils_rs.so Are you 100% sure you're comparing the exactly the same file?

fzyzcjy commented 2 years ago

Nice catch! vision_utils_rs is the executable (from main.rs) but libvision_utils_rs.so (notice lib and so) is the library (from lib.rs I guess) that is to be used. I will analyze the so now.

fzyzcjy commented 2 years ago

Now it sounds better, thank you!

[11:58:47]: $ cargo ndk --target armeabi-v7a -- bloat --release --crates --lib -n 1000
[11:58:47]: ▸ [2022-02-02T03:58:47Z INFO  cargo_ndk::cli] Using NDK at path: /Users/tom/Library/Android/sdk/ndk/21.2.6472646
[11:58:47]: ▸ [2022-02-02T03:58:47Z INFO  cargo_ndk::cli] NDK API level: 21
[11:58:47]: ▸ [2022-02-02T03:58:47Z INFO  cargo_ndk::cli] Building targets: armeabi-v7a
[11:58:47]: ▸ [2022-02-02T03:58:47Z INFO  cargo_ndk::cli] Building armeabi-v7a (armv7-linux-androideabi)
[11:58:49]: ▸ Finished release [optimized + debuginfo] target(s) in 1.25s
[11:58:49]: ▸ Analyzing target/armv7-linux-androideabi/release/libvision_utils_rs.so
[11:58:49]: ▸ File  .text     Size Crate
[11:58:49]: ▸ 1.0%  33.4%   1.8MiB [Unknown]
[11:58:49]: ▸ 0.3%  11.2% 628.4KiB std
[11:58:49]: ▸ 0.2%   7.8% 436.2KiB reqwest
[11:58:49]: ▸ 0.2%   7.0% 389.5KiB protobuf
[11:58:49]: ▸ 0.1%   3.1% 174.6KiB rustls
[11:58:49]: ▸ 0.1%   3.1% 171.6KiB vision_utils_rs
[11:58:49]: ▸ 0.1%   2.8% 157.8KiB ndarray
[11:58:49]: ▸ 0.1%   2.4% 135.9KiB h2
[11:58:49]: ▸ 0.1%   2.4% 134.9KiB sentry
[11:58:49]: ▸ 0.1%   2.3% 130.5KiB regex
[11:58:49]: ▸ 0.1%   2.1% 120.4KiB ring
[11:58:49]: ▸ 0.1%   2.1% 119.5KiB regex_syntax
[11:58:49]: ▸ 0.1%   1.7%  95.2KiB tokio
[11:58:49]: ▸ 0.0%   1.5%  86.6KiB sentry_core
[11:58:49]: ▸ 0.0%   1.5%  82.4KiB hyper
[11:58:49]: ▸ 0.0%   1.1%  63.0KiB backtrace
[11:58:49]: ▸ 0.0%   1.0%  55.6KiB hashbrown
[11:58:49]: ▸ 0.0%   0.9%  49.0KiB flutter_rust_bridge
[11:58:49]: ▸ 0.0%   0.8%  45.4KiB png
[11:58:49]: ▸ 0.0%   0.7%  40.9KiB rayon_core
[11:58:49]: ▸ 0.0%   0.7%  39.1KiB deflate
[11:58:49]: ▸ 0.0%   0.7%  37.7KiB rustc_demangle
[11:58:49]: ▸ 0.0%   0.6%  35.3KiB http
[11:58:49]: ▸ 0.0%   0.6%  34.7KiB opencv
[11:58:49]: ▸ 0.0%   0.6%  34.0KiB url
[11:58:49]: ▸ 0.0%   0.5%  25.6KiB image
[11:58:49]: ▸ 0.0%   0.4%  24.5KiB encoding_rs
[11:58:49]: ▸ 0.0%   0.4%  23.5KiB threadpool
[11:58:49]: ▸ 0.0%   0.4%  22.7KiB idna
[11:58:49]: ▸ 0.0%   0.4%  22.5KiB gimli
[11:58:49]: ▸ 0.0%   0.3%  18.7KiB sentry_backtrace
[11:58:49]: ▸ 0.0%   0.3%  18.2KiB aho_corasick
[11:58:49]: ▸ 0.0%   0.3%  16.4KiB anyhow
[11:58:49]: ▸ 0.0%   0.3%  15.8KiB rayon
[11:58:49]: ▸ 0.0%   0.3%  15.5KiB chrono
[11:58:49]: ▸ 0.0%   0.3%  15.5KiB miniz_oxide
[11:58:49]: ▸ 0.0%   0.3%  14.6KiB addr2line
[11:58:49]: ▸ 0.0%   0.3%  14.0KiB sentry_types
[11:58:49]: ▸ 0.0%   0.2%  12.2KiB sentry_contexts
[11:58:49]: ▸ 0.0%   0.2%  10.3KiB sentry_anyhow
[11:58:49]: ▸ 0.0%   0.2%  10.1KiB simplelog
[11:58:49]: ▸ 0.0%   0.2%   9.0KiB webpki
[11:58:49]: ▸ 0.0%   0.2%   8.7KiB delaunator
[11:58:49]: ▸ 0.0%   0.2%   8.5KiB sentry_panic
[11:58:49]: ▸ 0.0%   0.1%   8.0KiB parking_lot
[11:58:49]: ▸ 0.0%   0.1%   7.0KiB sentry_debug_images
[11:58:49]: ▸ 0.0%   0.1%   6.5KiB ryu
[11:58:49]: ▸ 0.0%   0.1%   4.8KiB memchr
[11:58:49]: ▸ 0.0%   0.1%   4.7KiB crossbeam_epoch
[11:58:49]: ▸ 0.0%   0.1%   4.4KiB httpdate
[11:58:49]: ▸ 0.0%   0.1%   4.0KiB bytes
[11:58:49]: ▸ 0.0%   0.1%   3.9KiB encoding
[11:58:49]: ▸ 0.0%   0.1%   3.9KiB indexmap
[11:58:49]: ▸ 0.0%   0.1%   3.5KiB ipnet
[11:58:49]: ▸ 0.0%   0.1%   3.2KiB mime
[11:58:49]: ▸ 0.0%   0.1%   3.2KiB tflite_rust
[11:58:49]: ▸ 0.0%   0.1%   3.0KiB tracing_core
[11:58:49]: ▸ 0.0%   0.1%   3.0KiB android_logger
[11:58:49]: ▸ 0.0%   0.0%   2.5KiB crossbeam_channel
[11:58:49]: ▸ 0.0%   0.0%   2.3KiB termcolor
[11:58:49]: ▸ 0.0%   0.0%   2.3KiB mio
[11:58:49]: ▸ 0.0%   0.0%   1.9KiB parking_lot_core
[11:58:49]: ▸ 0.0%   0.0%   1.9KiB httparse
[11:58:49]: ▸ 0.0%   0.0%   1.7KiB serde_json
[11:58:49]: ▸ 0.0%   0.0%   1.7KiB sct
[11:58:49]: ▸ 0.0%   0.0%   1.7KiB ndarray_stats
[11:58:49]: ▸ 0.0%   0.0%   1.6KiB findshlibs
[11:58:49]: ▸ 0.0%   0.0%   1.6KiB tracing
[11:58:49]: ▸ 0.0%   0.0%   1.5KiB unicode_normalization
[11:58:49]: ▸ 0.0%   0.0%   1.5KiB rand
[11:58:49]: ▸ 0.0%   0.0%   1.5KiB percent_encoding
[11:58:49]: ▸ 0.0%   0.0%   1.4KiB allo_isolate
[11:58:49]: ▸ 0.0%   0.0%   1.4KiB getrandom
[11:58:49]: ▸ 0.0%   0.0%   1.4KiB tokio_util
[11:58:49]: ▸ 0.0%   0.0%   1.3KiB base64
[11:58:49]: ▸ 0.0%   0.0%   1.2KiB tokio_rustls
[11:58:49]: ▸ 0.0%   0.0%   1.2KiB adler
[11:58:49]: ▸ 0.0%   0.0%   1.2KiB uuid
[11:58:49]: ▸ 0.0%   0.0%   1.1KiB crc32fast
[11:58:49]: ▸ 0.0%   0.0%   1.1KiB socket2
[11:58:49]: ▸ 0.0%   0.0%     996B crossbeam_deque
[11:58:49]: ▸ 0.0%   0.0%     936B jpeg_decoder
[11:58:49]: ▸ 0.0%   0.0%     788B once_cell
[11:58:49]: ▸ 0.0%   0.0%     752B debugid
[11:58:49]: ▸ 0.0%   0.0%     752B log
[11:58:49]: ▸ 0.0%   0.0%     744B adler32
[11:58:49]: ▸ 0.0%   0.0%     726B want
[11:58:49]: ▸ 0.0%   0.0%     678B uname
[11:58:49]: ▸ 0.0%   0.0%     578B pathfinding
[11:58:49]: ▸ 0.0%   0.0%     536B itertools
[11:58:49]: ▸ 0.0%   0.0%     510B env_logger
[11:58:49]: ▸ 0.0%   0.0%     344B time
[11:58:49]: ▸ 0.0%   0.0%     280B futures_core
[11:58:49]: ▸ 0.0%   0.0%     272B futures_util
[11:58:49]: ▸ 0.0%   0.0%     214B hostname
[11:58:49]: ▸ 0.0%   0.0%     196B object
[11:58:49]: ▸ 0.0%   0.0%     188B rand_chacha
[11:58:49]: ▸ 0.0%   0.0%     164B rand_core
[11:58:49]: ▸ 0.0%   0.0%     110B smallvec
[11:58:49]: ▸ 0.0%   0.0%     104B hex
[11:58:49]: ▸ 0.0%   0.0%      96B unicode_bidi
[11:58:49]: ▸ 0.0%   0.0%      74B futures_channel
[11:58:49]: ▸ 0.0%   0.0%      34B untrusted
[11:58:49]: ▸ 0.0%   0.0%      26B ppv_lite86
[11:58:49]: ▸ 0.0%   0.0%      26B futures_task
[11:58:49]: ▸ 0.0%   0.0%      24B spin
[11:58:49]: ▸ 0.0%   0.0%      16B num_cpus
[11:58:49]: ▸ 3.0% 100.0%   5.5MiB .text section size, the file size is 182.4MiB
[11:58:49]: ▸ Note: numbers above are a result of guesswork. They are not 100% correct and never will be.