search

bazelbuild / rules_rust

Rust rules for Bazel
https://bazelbuild.github.io/rules_rust/
Apache License 2.0
656 stars 419 forks source link

Updated fetch_shas utility to use the channel manifest tomls #2840

Closed UebelAndre closed 2 weeks ago

UebelAndre commented 3 weeks ago

Updating shas was taking much longer than it used to. I've refactored the shell script into python so to manage increased complexity. The script now fetches the channel-rust-*.toml files which contains additional sha256 values. This script still falls back to brute force fetching of artifacts to account for any that are missing. The runtime was reduced from ~20 min to ~80s.

python3.11 ./util/fetch_shas/fetch_shas.py
2024-09-06 08:57:22 - INFO - Fetching known sha256 data...
2024-09-06 08:57:22 - INFO - Downloading data...
2024-09-06 08:57:29 - INFO - Done.
2024-09-06 08:57:29 - INFO - Deserializing 119 tomls...
2024-09-06 08:57:36 - INFO - Done.
2024-09-06 08:57:36 - INFO - Parsing artifacts...
2024-09-06 08:57:36 - INFO - Done. Identified 52790 artifacts.
2024-09-06 08:57:36 - INFO - Checking for 81514 missing artifacts...
2024-09-06 08:58:26 - INFO - Done.
2024-09-06 08:58:29 - INFO - Downloading 404 missing artifacts...
2024-09-06 08:58:29 - INFO - Done.
2024-09-06 08:58:36 - INFO - Done. Wrote rust/known_shas.bzl

Note that beta sha256 values have been removed in this change.

UebelAndre commented 2 weeks ago

Thanks for fixing this all up! 🎉 🙏

I wonder if at some point we should remove versions before our MSRV from this list to speed up running the script? 🤷 We currently claim to support 1.72.1 and newer, so 64 of the 76 versions we currently fetch we claim not to support...

I'm down for removing some of the older things. Now that users are notified when they should add the shas (https://github.com/bazelbuild/rules_rust/pull/2593), I can open a PR for that.

UebelAndre commented 2 weeks ago

Thanks for fixing this all up! 🎉 🙏 I wonder if at some point we should remove versions before our MSRV from this list to speed up running the script? 🤷 We currently claim to support 1.72.1 and newer, so 64 of the 76 versions we currently fetch we claim not to support...

I'm down for removing some of the older things. Now that users are notified when they should add the shas (#2593), I can open a PR for that.

https://github.com/bazelbuild/rules_rust/pull/2850