Purge Python - Githubissues

Ezrashaw commented 1 year ago

Running the command: fd --extension py -E src/llvm-project gives a list of python files in rust-lang/rust:

compiler/rustc_codegen_gcc/tools/check_intrinsics_duplicates.py
compiler/rustc_codegen_gcc/tools/generate_intrinsics.py
library/core/src/unicode/printable.py
src/bootstrap/bootstrap.py
src/bootstrap/bootstrap_test.py
src/bootstrap/configure.py
src/ci/cpu-usage-over-time.py
src/ci/docker/host-x86_64/test-various/uefi_qemu_test/run.py
src/ci/docker/scripts/android-sdk-manager.py
src/ci/docker/scripts/fuchsia-test-runner.py
src/ci/github-actions/calculate-job-matrix.py
src/ci/scripts/upload-build-metrics.py
src/etc/dec2flt_table.py
src/etc/gdb_load_rust_pretty_printers.py
src/etc/gdb_lookup.py
src/etc/gdb_providers.py
src/etc/generate-deriving-span-tests.py
src/etc/generate-keyword-tests.py
src/etc/htmldocck.py
src/etc/lldb_batchmode.py
src/etc/lldb_lookup.py
src/etc/lldb_providers.py
src/etc/rust_types.py
src/tools/cargo/ci/generate.py
src/tools/cargo/publish.py
src/tools/clippy/util/versions.py
src/tools/miri/ci/scrape-targets.py
src/tools/miri/test-cargo-miri/run-test.py
src/tools/publish_toolstate.py
tests/debuginfo/auxiliary/dependency-with-embedded-visualizers.py
tests/debuginfo/embedded-visualizer-point.py
tests/debuginfo/embedded-visualizer.py
tests/run-make/debugger-visualizer-dep-info/my_gdb_script.py
tests/run-make/libtest-junit/validate_junit.py
x.py

While there will (probably) always be a need for a little bit of Python in bootstrap, we should aim to replace misc scripts that really should be Written In Rust. Especially relevant are files like src/etc/htmldocck.py which is the rustdoc UI test harness(!); we should rewrite this in Rust.

Extra points:

I wonder if CI scripts do actually need to be Python?
Longer term, I wonder if we can require a host installation of Rust (we just download one anyway) and use that for bootstrap?

Ezrashaw commented 1 year ago

cc @jyn514 am I crazy about getting rid of Python from bootstrap?

jyn514 commented 1 year ago

I don't see the point in rewriting all this code. The main goal of https://github.com/rust-lang/rust/issues/94829 is so we don't make people deal with python before they start working on the repo; but half the scripts you mentioned are for CI only. What do we get from rewriting them?

Ezrashaw commented 1 year ago

I guess maintainability in a way. I mean, why is compiletest not written in Python? It then makes sense to have the rustdoc part of it written in the same language, possibly src/etc/htmldocck.py could be rewritten and moved into compiletest itself. I'm not saying that this is important, just that it's something that should happen at some point.

jyn514 commented 1 year ago

I don't think this is worth spending contributor time on. There are other ways people can get started contributing (https://rustc-dev-guide.rust-lang.org/#what-should-i-work-on).

Ezrashaw commented 1 year ago

I agree, I think it's just good to have this here as a long-term goal.

compiler-errors commented 1 year ago

It's probably worthwhile to first step back and characterize the existing python that we have. I think only once it's clear what python actually can be rewritten should we actually discuss whether it should be rewritten.

e.g. those gdb and llvm scripts can't be rewritten in rust afaict.

Ezrashaw commented 1 year ago

Yeah, perhaps CI scripts as well might not be rewritable?

I do think there is some low-hanging fruit here which could have tangible benefits.

thomcc commented 1 year ago

I have a rewrite of library/core/src/unicode/printable.py on another machine, moving it into the rest of the table generation code. I'll try to get it up when I'm more easily able to contribute.

thomcc commented 1 year ago

Assigning to myself to do this for the unicode tables because @jyn514 indicated it might be the only one that's worth doing (no strong opinions from me, although I don't like that one being python)

Kobzol commented 2 months ago

Once Cargo script becomes stabilized, it should become feasible to turn some of these Python scripts into one-file Rust scripsts (of course only where it would make sense, there's no point in doing that proactively).

Zalathar commented 2 months ago

Of the remaining Python scripts in tests, most are gdb visualizer scripts, so trying to remove those doesn't make sense.

The one outlier is run-make/libtest-junit/validate_junit.py, which just takes advantage of Python's built-in XML parser to check for well-formed XML output. As noted in #129387, it's not worth pulling in a Rust XML crate just to get rid of one trivial Python script in one relatively-minor test.

(If we ever reach the point where building/testing the compiler without Python is feasible, we can just add a compiletest directive to skip that test if Python isn't present.)

Zalathar commented 2 months ago

Looking at the rest of the list, I mostly see:

More gdb scripts
CI scripts where there is presumably some value in being able to run independently of Rust
- EDIT: And in that case Python is better than Bash or similar
Infrequently-run code generator scripts that in principle should be ported to Rust, but where there's not much reason to actually port them unless people start running into specific maintenance headaches with the existing scripts

ChrisDenton commented 2 months ago

In terms of CI scripts, I think the choice is often Python or Bash. And I do prefer we use Python over Bash for anything non-trivial.

rust-lang / rust

Purge Python #110479