search

coffee-tools / coffee

Reference implementation for a flexible core lightning plugin manager
https://docs.page/coffee-tools/coffee
9 stars 13 forks source link

passing extra compilation flags with coffee #255

Open vincenzopalazzo opened 3 months ago

vincenzopalazzo commented 3 months ago

I think that we need a way to pass some custom parameters down the road to give some templating to pass down some env information

➜  ~ coffee install summars -v
   Compiling icu_datetime_data v1.4.0
   Compiling cln-rpc v0.1.8
   Compiling struct-field-names-as-array v0.3.0
   Compiling cln-plugin v0.1.8
error: could not compile `icu_datetime_data` (lib)

Caused by:
  process didn't exit successfully: `/home/vincent/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rustc --crate-name icu_datetime_data --edition=2021 /home/vincent/.cargo/registry/src/index.crates.io-6f17d22bba15001f/icu_datetime_data-1.4.0/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=104 --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no -C metadata=69d1c3cbabc16994 -C extra-filename=-69d1c3cbabc16994 --out-dir /home/vincent/.coffee/bitcoin/plugins/summars/target/release/deps -L dependency=/home/vincent/.coffee/bitcoin/plugins/summars/target/release/deps --cap-lints allow` (signal: 9, SIGKILL: kill)
warning: build failed, waiting for other jobs to finish...
    Building [=======================> ] 147/153: cln-rpc

The plugin info are there

➜  ~ coffee remote list
● List of repositories
╭──────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ ●   Repository Alias   URL                                       N. Plugins   Git HEAD   Last Update │
├──────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ●   folgore-git        https://github.com/coffee-tools/folgore   1            aedd703    12/03/2024  │
│ ●   summars            https://github.com/daywalker90/summars    1            c73633a    18/03/2024  │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────╯
vincenzopalazzo commented 3 months ago

now that I think a little bit maybe we can also pass an env variable like export JOBS=1 outside of coffee?

cc @daywalker90 because it is your plugin that it is OOM my machine :)

daywalker90 commented 3 months ago

Ahh how. I believe this is how you can measure memory usage during compilation:

cargo +nightly rustc --release --bin summars -- -Z time-passes
time:   0.001; rss:   39MB ->   42MB (   +3MB)  parse_crate
time:   0.000; rss:   42MB ->   47MB (   +4MB)  crate_injection
time:   0.032; rss:   47MB ->  138MB (  +91MB)  expand_crate
time:   0.032; rss:   47MB ->  138MB (  +91MB)  macro_expand_crate
time:   0.005; rss:  138MB ->  144MB (   +6MB)  late_resolve_crate
time:   0.000; rss:  144MB ->  147MB (   +3MB)  resolve_check_unused
time:   0.006; rss:  138MB ->  147MB (   +9MB)  resolve_crate
time:   0.006; rss:  147MB ->  151MB (   +4MB)  looking_for_entry_point
time:   0.008; rss:  147MB ->  151MB (   +4MB)  misc_checking_1
time:   0.002; rss:  151MB ->  155MB (   +4MB)  type_collecting
time:   0.013; rss:  155MB ->  176MB (  +21MB)  coherence_checking
time:   0.107; rss:  176MB ->  225MB (  +49MB)  wf_checking
time:   0.181; rss:  151MB ->  235MB (  +83MB)  type_check_crate
time:   0.060; rss:  235MB ->  243MB (   +9MB)  MIR_borrow_checking
time:   0.065; rss:  243MB ->  253MB (  +10MB)  MIR_effect_checking
time:   0.006; rss:  253MB ->  253MB (   +0MB)  misc_checking_3
time:   0.035; rss:  253MB ->  261MB (   +8MB)  monomorphization_collector_root_collections
time:   0.404; rss:  261MB ->  380MB ( +119MB)  monomorphization_collector_graph_walk
time:   0.128; rss:  380MB ->  398MB (  +18MB)  partition_and_assert_distinct_symbols
time:   0.000; rss:  398MB ->  398MB (   +0MB)  write_allocator_module
time:   1.726; rss:  398MB ->  750MB ( +352MB)  codegen_to_LLVM_IR
time:   3.597; rss:  253MB ->  750MB ( +497MB)  codegen_crate
time:   0.064; rss:  750MB ->  610MB ( -140MB)  free_global_ctxt
time:  18.231; rss:  470MB ->  540MB (  +70MB)  LLVM_passes
time:   0.001; rss:  539MB ->  514MB (  -25MB)  join_worker_thread
time:  15.419; rss:  610MB ->  514MB (  -96MB)  finish_ongoing_codegen
time:   0.000; rss:  514MB ->  513MB (   -1MB)  link_binary_check_files_are_writeable
time:   0.761; rss:  465MB ->  397MB (  -69MB)  run_linker
time:   0.765; rss:  514MB ->  397MB ( -117MB)  link_binary
time:   0.766; rss:  514MB ->  397MB ( -117MB)  link_crate
time:  16.185; rss:  610MB ->  397MB ( -213MB)  link
time:  20.702; rss:   31MB ->  128MB (  +97MB)  total

So for me it tops out at 750MB. For comparison coffee tops out at 321MB for me. But i don't know how to bring it down :(

vincenzopalazzo commented 2 months ago

Current workaround CARGO_BUILD_JOBS=1 coffee --network testnet install summars -v 😄