search

bschwind / opencascade-rs

Rust bindings to the OpenCascade CAD Kernel
GNU Lesser General Public License v2.1
118 stars 22 forks source link

CI: Add ccache support #110

Closed katyo closed 1 year ago

katyo commented 1 year ago

This is my attempt to speed-up OCCT build using ccache. Hope this help speed-up CI runs.

strohel commented 1 year ago

I've re-ran the Clippy and Test jobs to see how faster is the second (should be perfectly cached) run.

strohel commented 1 year ago

I've re-ran the Clippy and Test jobs to see how faster is the second (should be perfectly cached) run.

Hmm, the second runs don't seem to be any faster than the first ones? Also the baseline of the Test job seems to have increased from ~35 minutes to 45?

bschwind commented 1 year ago

My long-term plan for CI runs was:

And if that fails, we'd probably switch to dynamic linking in CI.

katyo commented 1 year ago

@strohel Seems something wrong because there is no cache hits on a second run: https://github.com/bschwind/opencascade-rs/actions/runs/5648868823/job/15304235571#step:20:5

katyo commented 1 year ago

@strohel I turned to using sccache but such behavior is not changed cardinally. It still has not enough cache hits to speed-up builds. https://github.com/bschwind/opencascade-rs/actions/runs/5652636438/job/15312573948#step:22:6 https://github.com/bschwind/opencascade-rs/actions/runs/5652636437/job/15312573945#step:22:6 Seems some unique onetime data goes to key on each run.

strohel commented 1 year ago

Yeah think (s)ccache might not be the best approach - because it caches on c++ call level, of which there are many. Would be better to cache the entire result of compilation (.a lib files), and that's what @bschwind's approach mentioned above would do.

katyo commented 1 year ago

@strohel It's so but it allows (in theory) partial rebuilts even if we upgraded OCCT sources.

I tested building occt-sys directly with sccache on my workstation:

[kayo@nixie-farm:~/devel/rust/opencascade-rs/crates/occt-sys]$ time cargo build
   Compiling libc v0.2.147
   Compiling cc v1.0.79
   Compiling either v1.9.0
   Compiling cmake v0.1.50
   Compiling which v4.4.0
   Compiling occt-sys v0.1.0 (/home/kayo/devel/rust/opencascade-rs/crates/occt-sys)
    Finished dev [unoptimized + debuginfo] target(s) in 4m 33s

real    4m33,459s
user    0m52,150s
sys 0m57,272s

[kayo@nixie-farm:~/devel/rust/opencascade-rs/crates/occt-sys]$ sccache -s
Compile requests                   5357
Compile requests executed          5357
Cache hits                           19
Cache hits (C/C++)                   19
Cache misses                       5338
Cache misses (C/C++)               5338
Cache timeouts                        0
Cache read errors                     0
Forced recaches                       0
Cache write errors                    0
Compilation failures                  0
Cache errors                          0
Non-cacheable compilations            0
Non-cacheable calls                   0
Non-compilation calls                 0
Unsupported compiler calls            0
Average cache write               0.000 s
Average compiler                  1.028 s
Average cache read hit            0.000 s
Failed distributed compilations       0
Cache location                  Local disk: "/home/kayo/.cache/sccache"
Version (client)                0.4.2
Cache size                          499 MiB
Max cache size                       10 GiB

[kayo@nixie-farm:~/devel/rust/opencascade-rs/crates/occt-sys]$ cargo clean

[kayo@nixie-farm:~/devel/rust/opencascade-rs/crates/occt-sys]$ time cargo build
   Compiling libc v0.2.147
   Compiling cc v1.0.79
   Compiling either v1.9.0
   Compiling cmake v0.1.50
   Compiling which v4.4.0
   Compiling occt-sys v0.1.0 (/home/kayo/devel/rust/opencascade-rs/crates/occt-sys)
    Finished dev [unoptimized + debuginfo] target(s) in 45.41s

real    0m45,428s
user    0m44,033s
sys 0m50,644s

[kayo@nixie-farm:~/devel/rust/opencascade-rs/crates/occt-sys]$ sccache -s
Compile requests                  10714
Compile requests executed         10714
Cache hits                         5376
Cache hits (C/C++)                 5376
Cache misses                       5338
Cache misses (C/C++)               5338
Cache timeouts                        0
Cache read errors                     0
Forced recaches                       0
Cache write errors                    0
Compilation failures                  0
Cache errors                          0
Non-cacheable compilations            0
Non-cacheable calls                   0
Non-compilation calls                 0
Unsupported compiler calls            0
Average cache write               0.000 s
Average compiler                  1.028 s
Average cache read hit            0.000 s
Failed distributed compilations       0
Cache location                  Local disk: "/home/kayo/.cache/sccache"
Version (client)                0.4.2
Cache size                          499 MiB
Max cache size                       10 GiB

As you can see cache works fine. Same results when building opencascade crate.

[kayo@nixie-farm:~/devel/rust/opencascade-rs/crates/opencascade]$ time cargo build
   Compiling cc v1.0.79
   Compiling proc-macro2 v1.0.66
   Compiling unicode-ident v1.0.11
   Compiling libc v0.2.147
   Compiling either v1.9.0
   Compiling scratch v1.0.7
   Compiling termcolor v1.2.0
   Compiling unicode-width v0.1.10
   Compiling cxxbridge-flags v1.0.102
   Compiling once_cell v1.18.0
   Compiling thiserror v1.0.44
   Compiling bytemuck v1.13.1
   Compiling glam v0.23.0
   Compiling codespan-reporting v0.11.1
   Compiling cmake v0.1.50
   Compiling quote v1.0.32
   Compiling link-cplusplus v1.0.9
   Compiling cxx v1.0.102
   Compiling which v4.4.0
   Compiling syn v2.0.27
   Compiling occt-sys v0.1.0 (/home/kayo/devel/rust/opencascade-rs/crates/occt-sys)
   Compiling cxx-build v1.0.102
   Compiling cxxbridge-macro v1.0.102
   Compiling thiserror-impl v1.0.44
   Compiling opencascade-sys v0.1.0 (/home/kayo/devel/rust/opencascade-rs/crates/opencascade-sys)
   Compiling opencascade v0.1.0 (/home/kayo/devel/rust/opencascade-rs/crates/opencascade)
    Finished dev [unoptimized + debuginfo] target(s) in 4m 39s

real    4m39,254s
user    1m11,471s
sys 0m59,868s

[kayo@nixie-farm:~/devel/rust/opencascade-rs/crates/opencascade]$ sccache -s
Compile requests                   5357
Compile requests executed          5357
Cache hits                           19
Cache hits (C/C++)                   19
Cache misses                       5338
Cache misses (C/C++)               5338
Cache timeouts                        0
Cache read errors                     0
Forced recaches                       0
Cache write errors                    0
Compilation failures                  0
Cache errors                          0
Non-cacheable compilations            0
Non-cacheable calls                   0
Non-compilation calls                 0
Unsupported compiler calls            0
Average cache write               0.000 s
Average compiler                  1.030 s
Average cache read hit            0.000 s
Failed distributed compilations       0
Cache location                  Local disk: "/home/kayo/.cache/sccache"
Version (client)                0.4.2
Cache size                          499 MiB
Max cache size                       10 GiB

[kayo@nixie-farm:~/devel/rust/opencascade-rs/crates/opencascade]$ cargo clean

[kayo@nixie-farm:~/devel/rust/opencascade-rs/crates/opencascade]$ time cargo build
   Compiling cc v1.0.79
   Compiling proc-macro2 v1.0.66
   Compiling unicode-ident v1.0.11
   Compiling libc v0.2.147
   Compiling either v1.9.0
   Compiling scratch v1.0.7
   Compiling unicode-width v0.1.10
   Compiling termcolor v1.2.0
   Compiling cxxbridge-flags v1.0.102
   Compiling once_cell v1.18.0
   Compiling thiserror v1.0.44
   Compiling bytemuck v1.13.1
   Compiling glam v0.23.0
   Compiling codespan-reporting v0.11.1
   Compiling cmake v0.1.50
   Compiling quote v1.0.32
   Compiling link-cplusplus v1.0.9
   Compiling cxx v1.0.102
   Compiling which v4.4.0
   Compiling syn v2.0.27
   Compiling occt-sys v0.1.0 (/home/kayo/devel/rust/opencascade-rs/crates/occt-sys)
   Compiling cxx-build v1.0.102
   Compiling cxxbridge-macro v1.0.102
   Compiling thiserror-impl v1.0.44
   Compiling opencascade-sys v0.1.0 (/home/kayo/devel/rust/opencascade-rs/crates/opencascade-sys)
   Compiling opencascade v0.1.0 (/home/kayo/devel/rust/opencascade-rs/crates/opencascade)
    Finished dev [unoptimized + debuginfo] target(s) in 54.23s

real    0m54,241s
user    1m2,245s
sys 0m51,096s

[kayo@nixie-farm:~/devel/rust/opencascade-rs/crates/opencascade]$ sccache -s
Compile requests                  10714
Compile requests executed         10714
Cache hits                         5376
Cache hits (C/C++)                 5376
Cache misses                       5338
Cache misses (C/C++)               5338
Cache timeouts                        0
Cache read errors                     0
Forced recaches                       0
Cache write errors                    0
Compilation failures                  0
Cache errors                          0
Non-cacheable compilations            0
Non-cacheable calls                   0
Non-compilation calls                 0
Unsupported compiler calls            0
Average cache write               0.000 s
Average compiler                  1.030 s
Average cache read hit            0.000 s
Failed distributed compilations       0
Cache location                  Local disk: "/home/kayo/.cache/sccache"
Version (client)                0.4.2
Cache size                          499 MiB
Max cache size                       10 GiB

Seems something happens wrong on CI.

bschwind commented 1 year ago

@katyo I have CI build times down to a couple minutes now, see #91

Okay if I close this?