search

RazrFalcon / cargo-bloat

Find out what takes most of the space in your executable.
MIT License
2.33k stars 50 forks source link

cargo bloat --time can't execute cargo build #73

Closed totikom closed 3 years ago

totikom commented 3 years ago

Steps to reproduce

  1. clone solana repo.
  2. run cargo bloat --time

Output

image This repo can be builded with cargo build with no errors.

Meanwhile bare cargo bloat works.

RazrFalcon commented 3 years ago

Works fine for me:

> cargo bloat --time
  Time Crate
14.38s solana_runtime
11.64s libsecp256k1
11.49s libsecp256k1
10.59s syn
10.28s syn
 8.63s syn
 7.30s bindgen
 7.02s reqwest
 6.74s hyper
 6.54s serde_derive
 6.42s tokio
 5.99s tonic
 5.85s syn
 5.85s goblin
 5.55s solana_client
 5.36s serde
 5.29s solana_program
 5.06s clang_sys
 5.05s h2
 5.04s librocksdb_sys
 4.95s solana_cli
 4.92s clap
 4.78s serde
 4.69s clap
 4.51s tokio
 4.46s libc
 4.44s solana_program
 4.22s etcd_client
 4.15s solana
 4.11s rustls
 4.09s jsonrpc_derive
 4.08s jsonrpc_derive
 4.01s derive_more
 3.97s regex
 3.97s solana_stake_accounts
 3.90s solana_sdk
 3.89s solana_tokens
 3.75s libc
 3.75s solana_rbpf
 3.57s solana_install
 3.55s regex_syntax
 3.46s cargo_build_bpf
 3.42s combine
 3.40s solana_accounts_bench
 3.38s solana_watchtower
 3.36s rbpf_cli
 3.33s solana_bpf_loader_program
 3.25s solana_storage_proto
 3.23s encoding_rs
 3.12s solana_install_init
 3.04s proc_macro2
 3.02s hyper
 3.02s ring
 2.99s proc_macro2
 2.96s nix
 2.94s openssl
 2.88s object
 2.81s solana_transaction_status
 2.68s opentelemetry
 2.68s pin_project_internal
 2.66s solana_cli_output
 2.63s raptorq
 2.60s derivative
 2.52s solana_faucet
 2.48s futures_util
 2.45s prost_derive
 2.40s solana_banks_server
 2.39s pin_project_internal
 2.38s prost_build
 2.38s solana_account_decoder
 2.26s gimli
 2.24s solana_tokens
 2.17s async_trait
 2.16s websocket
 2.14s solana_vote_program
 2.12s solana_poh_bench
 2.07s solana_bench_streamer
 2.07s memchr
 2.07s solana_storage_bigtable
 2.06s clap_derive
 2.06s toml
 2.06s nom
 2.05s solana_upload_perf
 2.05s cargo_metadata
 2.05s educe
 2.01s serde_json
 1.98s solana_entry
 1.96s solana_merkle_root_bench
 1.94s ouroboros_macro
 1.93s solana_keygen
 1.93s solana_install
 1.88s blake3
 1.88s prost_types
 1.87s tarpc
 1.84s blake3
 1.83s solana_store_tool
 1.82s num_bigint
 1.82s prost_types
 1.81s rayon
 1.78s bitvec
 1.77s time
 1.75s log
 1.73s log
 1.70s log
 1.66s tracing_attributes
 1.63s futures
 1.63s backtrace
 1.59s rayon_core
 1.58s tarpc_plugins
 1.57s solana_net_utils
 1.57s trees
 1.55s proc_macro_hack
 1.55s num_derive
 1.55s cc
 1.48s thiserror_impl
 1.48s solana_program_test
 1.44s solana_log_analyzer
 1.44s const_format_proc_macros
 1.43s failure_derive
 1.43s solana_perf
 1.42s tower
 1.41s borsh
 1.40s libsecp256k1_core
 1.39s futures_macro
 1.38s globset
 1.37s num_enum_derive
 1.37s pickledb
 1.35s num_traits
 1.35s curve25519_dalek
 1.35s num_traits
 1.35s rand
 1.35s cc
 1.34s semver
 1.33s rand
 1.32s chrono
 1.31s solana_ledger
 1.30s rustversion
 1.30s generic_array
 1.29s security_framework
 1.29s jsonrpc_core
 1.29s tar
 1.29s jsonrpc_core
 1.28s tungstenite
 1.28s http
 1.26s curve25519_dalek
 1.26s cargo_test_bpf
 1.25s semver
 1.25s mio
 1.24s bv
 1.24s solana_faucet
 1.23s solana_sdk_macro
 1.22s solana_stake_program
 1.21s parity_ws
 1.20s solana_net_shaper
 1.19s serde_yaml
 1.18s uriparse
 1.17s scroll_derive
 1.17s num_enum
 1.16s tonic_build
 1.16s rand
 1.16s indexmap
 1.16s miniz_oxide
 1.16s dialoguer
 1.15s time_macros_impl
 1.14s solana_ip_address_server
 1.14s async_stream_impl
 1.14s solana_frozen_abi_macro
 1.14s url
 1.14s solana_frozen_abi_macro
 1.13s goauth
 1.13s reed_solomon_erasure
 1.13s jsonrpc_client_transports
 1.12s solana_sys_tuner
 1.09s solana_rpc
 1.07s num_integer
 1.07s tokio_macros
 1.07s url
 1.06s rand
 1.06s crossbeam_utils
 1.06s prost
 1.05s jsonrpc_http_server
 1.04s crossbeam_utils
 1.02s const_fn
 1.02s solana_ip_address
 1.01s rand
 1.01s mio
 1.01s indicatif
 1.01s semver
 1.01s jsonrpc_server_utils
 1.01s time
 1.01s solana_metrics
 0.99s tracing_subscriber
 0.99s jsonrpc_server_utils
 0.99s aho_corasick
 0.99s synstructure
 0.98s cexpr
 0.98s proc_macro_error_attr
 0.97s num_integer
 0.96s httparse
 0.95s tokio_macros
 0.95s indexmap
 0.95s enum_ordinalize
 0.94s scroll
 0.94s solana_remote_wallet
 0.93s tracing
 0.93s getrandom
 0.92s getrandom
 0.92s hidapi
 0.92s solana_sdk_macro
 0.92s time
 0.91s camino
 0.91s getrandom
 0.90s time_macros
 0.90s idna
 0.90s jsonrpc_client_transports
 0.90s unicode_normalization
 0.89s proc_macro_error
 0.89s petgraph
 0.88s idna
 0.88s zeroize_derive
 0.88s bip39
 0.87s libsecp256k1_core
 0.87s console
 0.86s serde_cbor
 0.86s solana_clap_utils
 0.84s anyhow
 0.83s jsonrpc_pubsub
 0.82s parking_lot
 0.82s spl_token
 0.82s generic_array
 0.82s jsonrpc_pubsub
 0.82s borsh_derive
 0.81s env_logger
 0.81s enum_iterator_derive
 0.81s jsonrpc_ws_server
 0.81s winapi
 0.81s solana_ownable
 0.80s bip39
 0.80s dlopen_derive
 0.80s bstr
 0.79s itertools
 0.79s solana_config_program
 0.79s users
 0.79s prost
 0.78s funty
 0.78s env_logger
 0.78s csv
 0.78s parking_lot
 0.78s ipnet
 0.77s language_tags
 0.77s itertools
 0.77s tokio_util
 0.76s parking_lot_core
 0.76s yaml_rust
 0.74s zstd_sys
 0.74s pin_project
 0.72s build_script_main
 0.71s async_stream
 0.70s console
 0.70s crossbeam_epoch
 0.70s unicase
 0.69s native_tls
 0.69s jsonrpc_ipc_server
 0.69s solana_banks_interface
 0.69s solana_frozen_abi
 0.68s pin_project
 0.68s futures_executor
 0.68s generic_array
 0.67s solana_frozen_abi
 0.67s pkg_config
 0.67s crossbeam_channel
 0.67s ryu
 0.66s typenum
 0.65s tokio_reactor
 0.64s thiserror
 0.64s standback
 0.64s signal_hook
 0.64s sysctl
 0.64s solana_noop_program
 0.64s solana_streamer
 0.64s solana_csv_to_validator_infos
 0.63s websocket_base
 0.63s tinyvec
 0.63s bytes
 0.63s standback
 0.62s build_script_main
 0.62s crc32fast
 0.62s bytes
 0.61s unicase
 0.61s signal_hook_registry
 0.60s bincode
 0.60s bytes
 0.60s ahash
 0.59s solana_core
 0.58s bytes
 0.58s glob
 0.58s solana_gossip
 0.57s const_format
 0.56s parking_lot_core
 0.55s futures_channel
 0.55s flate2
 0.55s radium
 0.55s solana_version
 0.55s spl_memo
 0.55s inflector
 0.55s zstd_safe
 0.54s webpki
 0.54s failure
 0.54s jsonrpc_ipc_server
 0.54s itertools
 0.54s core_foundation
 0.54s ahash
 0.54s tokio_io
 0.54s unicode_segmentation
 0.54s proc_macro_crate
 0.53s cargo_platform
 0.53s futures_task
 0.53s semver_parser
 0.51s socket2
 0.51s tokio_stream
 0.51s strsim
 0.51s solana_download_utils
 0.51s core_foundation_sys
 0.50s spl_associated_token_account
 0.50s enum_iterator
 0.50s zeroize
 0.50s rocksdb
 0.50s borsh_schema_derive_internal
 0.49s rustc_demangle
 0.49s rustc_version
 0.49s solana_notifier
 0.49s autocfg
 0.49s jobserver
 0.49s half
 0.49s autocfg
 0.49s jobserver
 0.48s borsh_derive_internal
 0.48s sha2
 0.48s futures_core
 0.48s proc_macro_nested
 0.48s solana_cli_config
 0.47s maybe_uninit
 0.47s termcolor
 0.47s bzip2_sys
 0.46s quote
 0.46s solana_gossip
 0.46s version_check
 0.46s rand_hc
 0.45s openssl_sys
 0.45s rustc_version
 0.44s unicode_bidi
 0.44s strsim
 0.44s crunchy
 0.43s dlopen
 0.43s tokio_util
 0.43s ascii
 0.43s net2
 0.43s version_check
 0.43s quote
 0.43s mime
 0.43s rand_chacha
 0.42s tracing_core
 0.42s base64
 0.42s hashbrown
 0.42s base64
 0.42s cast
 0.42s sha3
 0.42s mime
 0.41s fs_extra
 0.41s humantime
 0.41s chrono_humanize
 0.41s csv_core
 0.41s sha2
 0.41s hashbrown
 0.40s tempfile
 0.40s zstd
 0.40s rand_chacha
 0.40s tracing_opentelemetry
 0.40s tempfile
 0.39s rand_chacha
 0.39s ed25519_dalek
 0.39s rand_isaac
 0.39s unix_socket
 0.38s byteorder
 0.38s hex
 0.38s serde_urlencoded
 0.38s tokio_executor
 0.38s sharded_slab
 0.38s rand_chacha
 0.38s dashmap
 0.38s tokio_serde
 0.38s ppv_lite86
 0.37s solana_merkle_tree
 0.36s rand_pcg
 0.36s arrayvec
 0.36s solana_banks_client
 0.36s filedescriptor
 0.36s arc_swap
 0.36s derivation_path
 0.36s memoffset
 0.36s tokio_sync
 0.35s tokio_tcp
 0.35s rand_pcg
 0.35s inflector
 0.35s quote
 0.34s httpdate
 0.34s xattr
 0.34s mio_extras
 0.34s rand_core
 0.33s sha1
 0.33s solana_logger
 0.33s ouroboros
 0.33s hyper_tls
 0.33s smpl_jwt
 0.33s rpassword
 0.33s bzip2
 0.33s parity_tokio_ipc
 0.32s rand_core
 0.32s solana_failure_program
 0.32s fixedbitset
 0.32s which
 0.32s memmap2
 0.32s solana_genesis_utils
 0.31s core_affinity
 0.31s ed25519_dalek_bip32
 0.31s ansi_term
 0.30s walkdir
 0.30s arc_swap
 0.30s backoff
 0.30s sha1
 0.30s histogram
 0.30s serde_bytes
 0.29s parity_tokio_ipc
 0.29s gag
 0.29s sha1
 0.29s base32
 0.29s criterion_stats
 0.29s rand_core
 0.28s solana_logger
 0.28s gethostname
 0.28s bs58
 0.28s thread_local
 0.27s bs58
 0.27s smallvec
 0.27s percent_encoding
 0.27s filetime
 0.26s shlex
 0.26s utf8
 0.26s libloading
 0.26s solana_ledger_udev
 0.26s base64
 0.26s ctrlc
 0.26s rand_xorshift
 0.26s tokio_io_timeout
 0.26s feature_probe
 0.26s rand_jitter
 0.25s percent_encoding
 0.25s memmap2
 0.25s sct
 0.25s mio_uds
 0.25s http_body
 0.25s rand_core
 0.25s futures_io
 0.24s base64
 0.24s want
 0.24s rand_os
 0.24s iovec
 0.24s smallvec
 0.24s keccak
 0.24s tokio_rustls
 0.23s textwrap
 0.23s libsecp256k1_gen_genmult
 0.22s textwrap
 0.22s dirs_sys_next
 0.22s hashbrown
 0.22s adler
 0.22s solana_genesis
 0.22s solana_measure
 0.22s crossbeam_deque
 0.22s either
 0.21s regex_automata
 0.21s libsecp256k1_gen_ecmult
 0.21s ed25519
 0.21s same_file
 0.21s qstring
 0.21s digest
 0.21s once_cell
 0.21s solana_ledger_tool
 0.20s dtoa
 0.20s solana_rayon_threadlimit
 0.20s security_framework_sys
 0.20s tokio_codec
 0.20s bytecount
 0.20s hyper_timeout
 0.19s solana_sys_tuner
 0.19s arrayvec
 0.19s heck
 0.19s hmac
 0.19s unicode_xid
 0.19s block_padding
 0.19s solana_genesis
 0.18s os_str_bytes
 0.18s untrusted
 0.18s aliasable
 0.18s number_prefix
 0.18s solana_dos
 0.18s dir_diff
 0.18s termios
 0.18s subtle
 0.18s block_buffer
 0.18s tokio_native_tls
 0.18s solana_replica_node
 0.18s linked_hash_map
 0.17s vec_map
 0.17s crypto_mac
 0.17s itoa
 0.17s addr2line
 0.17s solana_accounts_cluster_bench
 0.17s solana_test_validator
 0.17s lru
 0.17s futures_sink
 0.17s hyper_rustls
 0.17s solana_banking_bench
 0.17s num_cpus
 0.16s crypto_mac
 0.16s digest
 0.16s either
 0.16s crypto_mac
 0.16s hmac
 0.16s form_urlencoded
 0.16s hash32
 0.16s digest
 0.16s subtle
 0.16s solana_compute_budget_program
 0.16s byte_unit
 0.16s tracing_futures
 0.16s cpufeatures
 0.15s constant_time_eq
 0.15s wyz
 0.15s tower_layer
 0.15s slab
 0.15s webpki_roots
 0.15s ieee754
 0.15s terminal_size
 0.15s solana_validator
 0.15s fd_lock
 0.15s spin
 0.14s tokio_tls
 0.14s multimap
 0.14s dirs_next
 0.14s lock_api
 0.14s safemem
 0.14s signature
 0.14s lock_api
 0.14s hmac
 0.14s pbkdf2
 0.14s jsonrpc_core_client
 0.14s try_lock
 0.14s scopeguard
 0.14s plain
 0.14s lazycell
 0.13s pbkdf2
 0.13s jsonrpc_core_client
 0.13s pbkdf2
 0.13s block_buffer
 0.13s unicode_xid
 0.13s crypto_mac
 0.13s solana_secp256k1_program
 0.12s hmac
 0.12s hmac_drbg
 0.12s pretty_hex
 0.12s lazy_static
 0.12s solana_validator
 0.12s solana_bench_tps
 0.12s solana_local_cluster
 0.12s cfg_if
 0.12s unicode_width
 0.11s cfg_if
 0.11s thread_scoped
 0.11s atty
 0.11s fake_simd
 0.11s solana_poh
 0.11s rustc_hash
 0.11s traitobject
 0.11s peeking_take_while
 0.11s tap
 0.11s pin_project_lite
 0.11s futures
 0.11s tower_service
 0.11s lazy_static
 0.11s block_padding
 0.11s instant
 0.10s void
 0.10s solana_replica_node
 0.10s arrayref
 0.10s byte_tools
 0.10s foreign_types_shared
 0.10s retain_mut
 0.10s simpl
 0.10s pin_utils
 0.10s miow
 0.10s mio_named_pipes
 0.10s static_assertions
 0.10s fnv
 0.10s symlink
 0.10s pin_project_lite
 0.09s matches
 0.09s typeable
 0.09s solana_crate_features
 0.09s solana_bench_tps
 0.09s utf8_width
 0.09s stable_deref_trait
 0.09s bitflags
 0.09s opaque_debug
 0.08s opaque_debug
 0.08s fast_math
 0.08s tinyvec_macros
 0.08s assert_matches
 0.08s remove_dir_all
 0.07s unreachable
 0.07s rand_core
 0.07s foreign_types

The error message is definitely should be improved.

RazrFalcon commented 3 years ago

Turns out master is actually broken.

> cargo build --release
   Compiling solana-ledger v1.8.0 (/Users/razr/Downloads/solana/ledger)
error[E0658]: use of unstable library feature 'map_into_keys_values'
   --> ledger/src/blockstore.rs:935:50
    |
935 |         for shred in just_inserted_coding_shreds.into_values() {
    |                                                  ^^^^^^^^^^^
    |
    = note: see issue #75294 <https://github.com/rust-lang/rust/issues/75294> for more information

error: aborting due to previous error

For more information about this error, try `rustc --explain E0658`.
error: could not compile `solana-ledger`

To learn more, run the command again with --verbose.

So in my case --time is fine, somehow, but cargo bloat --release fails.

totikom commented 3 years ago

Quite strange, as on my machine cargo build succeeded. Unfortunately, I've git pull'ed master, to recheck my results (now it is consistent with yours), so I can't point to the exact commit (there are 50+ commits since my last pull), where the problem was found.

I'll close this issue as unreproducible.

trevor-crypto commented 3 years ago

cargo bloat --time just hangs for me on the latest crates.io release. (i.e I did cargo install cargo-bloat). Going to try from master too

RazrFalcon commented 3 years ago

--time just a RUSTC_WRAPPER. It doesn't do anything by itself.

trevor-crypto commented 3 years ago

What do you mean? I thought the correct way to run bloat is cargo bloat --time and that's all? I enter that and it just hangs and I get no feedback/output. Any thing I should try before adding debug lines?

totikom commented 3 years ago

@trevor-crypto it is possible, that cargo bloat --time not hangs, but working too long. Internally, it runst cargo build without usual status output. That means, that if your crate compiles more than an hour, cargo bloat --time will be running more than an hour.

totikom commented 3 years ago

Speaking of that, @RazrFalcon, in process_crate() function stderr is deliberately piped, as it is used to calculate build times. https://github.com/RazrFalcon/cargo-bloat/blob/7d3ce2167d5b4e8d27bc058d11bd6c0afde40538/src/main.rs#L527-L533

Is it possible to continuously read data from stderr of the child process and print it to our stderr? Doing so, we will show information about the invoked cargo build to the user.

If I figure out a way to do it, I'll create a pull request.

RazrFalcon commented 3 years ago

Yes, it hides the output. Don't remember why exactly. You can fiddle around it with it and send me a pull request if you will be able to avoid it.